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EXECUTIVE  SUMMARY 


This  document  reports  on  the  ESTCP  live-site  demonstration  that  was  performed  with  the 
Man-Portable  Vector  (MPV)  sensor  at  the  Spencer  Artillery  Range  near  Spencer,  Tennessee  in 
June  2012.  This  study  encompasses  the  data  collection  survey,  data  processing  and  classification 
and  performance  analysis. 

The  MPV  technology  is  a  handheld  sensor  for  munitions  detection  and  classification.  Its  form 
factor  was  specifically  designed  for  sites  such  as  Spencer  Range,  where  vegetation  or  terrain 
limits  access  to  vehicle-based  platforms.  The  MPV  is  an  electromagnetic  induction  (EMI)  sensor 
that  comprises  of  a  50-centimeter  diameter  transmitter  and  an  array  of  three-dimensional 
receivers.  The  MPV  can  be  deployed  with  multiple  positioning  systems  depending  on  the 
environment.  Standard  GPS  technology  can  be  used  under  open  sky.  Under  thick  canopy  GPS 
signal  is  disturbed  and  therefore  a  specific  portable  local  positioning  system  was  developed.  Its 
principle  is  based  on  locating  the  MPV  transmitter,  which  acts  similar  to  a  beacon,  and  is  not 
affected  by  natural  obstacles.  A  survey  can  therefore  be  performed  in  forested  and  rugged 
environments.  The  MPV  can  be  employed  for  detection  survey  and  for  cued  interrogation,  in 
which  high  quality  data  are  acquired  near  a  target  to  achieve  the  best  classification.  The  MPV 
technology  was  validated  at  Yuma  Proving  Ground,  Arizona  UXO  Standardized  Test  Site  in 
October  2010,  where  90%  of  targets  within  1-m  depth  were  detected  and  correctly  classified.  The 
sensor  was  subsequently  demonstrated  at  former  Camp  Beale,  California  in  June  2011  along 
other  portable  systems  in  a  treed  environment,  achieving  100%  correct  classification. 

The  Spencer  study  was  designed  to  test  portable  technologies  along  with  vehicular  systems. 
A  forested  area  contained  700  flags  that  were  characterized  by  cued  interrogation  with  the 
portable  sensors,  whereas  a  1.3-acre  open  area  hosting  approximately  300  potential  targets  was 
tested  with  most  systems  doing  a  detection  survey  followed  with  cued  interrogation  of  the 
detected  anomalies.  A  testing  lane  and  empty  pit  were  set  up  for  calibrating  the  MPV  over 
known  targets  every  day.  The  MPV  was  deployed  with  one  experienced  and  one  new  field 
operator,  and  one  on-site  data  processor.  Dynamic  detection  data  were  collected  in  2.5  days;  data 
were  mapped  and  interpreted  to  select  anomalies  that  were  validated  by  the  ESTCP  for  further 
characterization.  The  1000  anomalies  of  interest  were  acquired  over  8  days.  Some  days  as  many 
as  150  anomalies  were  visited.  The  entire  survey  was  completed  without  any  technical 
interruption.  Data  were  reviewed  and  quality-controlled  on-site;  after  collection,  pre-processed 
data  were  distributed  to  other  ESTCP  analysts  in  support  of  the  classification  study.  Our  study 
was  divided  in  two  parts.  For  the  forested  area,  cued  data  analysis  allowed  99%  correct 
classification  of  buried  munitions  while  rejecting  85%  of  the  clutter.  The  dynamic  area  was 
treated  using  dynamic  data  as  a  pre-screener  to  classify  half  of  the  anomalies,  and  turning  to 
cued  data  for  the  more  ambiguous  anomalies.  All  targets  of  interest  were  found  while  rejecting 
90%  of  the  clutter,  which  indicates  a  strong  potential  for  significantly  reducing  the  number  of 
anomalies  that  need  to  be  investigated  in  cued  mode  for  future  studies.  All  demonstration 
objectives  were  successfully  met. 

The  MPV  has  now  been  demonstrated  at  one  proving  ground  and  two  live  sites  where 
classification  has  been  successful  and  field  productivity  has  improved.  The  site  conditions  posed 
no  particular  difficulties  for  a  handheld  technology  like  the  MPV;  other  sites  will  be  visited  to 
further  establish  the  performance,  limitations,  optimum  usage  and  costs  of  the  technology.  The 
MPV  is  scheduled  to  be  deployed  and  tested  at  several  sites  through  year  2013  as  part  of  the 
ESTCP  ongoing  live-site  demonstration  program. 
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1.0  INTRODUCTION 


The  demonstration  at  former  Spencer  Artillery  Range  (Spencer)  is  one  in  the  series  of 
Environmental  Security  Technology  Certification  Program  (ESTCP)  demonstrations  of 
classification  technologies  for  Munitions  Response  (MR).  This  demonstration  is  designed  to 
investigate  the  evolving  classification  methodology  at  a  site  that  is  partially  wooded  with  a  mix 
of  munitions  types.  This  project  proposes  to  demonstrate  detection  and  classification  with  the 
Man  Portable  Vector  (MPV)  sensor.  The  MPV  is  an  electromagnetic  induction  (EMI)  sensor  that 
was  designed  to  extend  advanced  discrimination  capabilities  to  sites  with  challenging  surveying 
conditions  and,  thus  allow  for  advanced  discrimination  to  be  applied  at  most  human  trafficable 
land  locations  at  moderate  cost.  The  system  is  deployed  in  dynamic  search  to  test  detection 
capabilities  and  in  cued  interrogation,  where  a  series  of  detected  EMI  anomalies  are  investigated 
for  classification  by  collecting  multiple  static  soundings. 

In  the  following  document,  we  shall  try  to  abide  by  the  following  definitions.  An  anomaly  is 
a  signal  or  a  location  where  a  sensor  detected  a  peak  in  the  EMI  data  -  in  that  sense  it  is  correct  to 
refer  to  inverted  or  classified  anomalies.  The  flag  is  the  marked  location  where  the  detection 
survey  predicted  the  anomaly  to  be  located.  Each  flag  is  associated  with  a  label  that  indicates  the 
site  specific  identity  number.  A  target  of  interest  (TOI)  is  an  ammunition  or  a  seeded  simulant 
such  as  an  ISO  cylinder.  Target  picking  is  the  action  of  selecting  anomalies  of  interest. 


2.0  TECHNOLOGY 

The  MPV  technology  is  based  on  electromagnetic  induction  sensing  using  one  transmitter 
coil  and  multiple  vector  receivers  in  a  handheld  form  factor.  The  sensor  presented  in  this  study  is 
the  second-generation  prototype  MPV. 


2.1  MPV  TECHNOLOGY  DESCRIPTION 
2.1.1  Electromagnetic  sensor 
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Figure  1:  The  MPV  as  deployed  at  Spencer  Range,  TN  in  June  2012. 

It  includes  a  sensor  head  with  one  transmitter  loop  and  five  3D  receiver  cubes  inside  a  transparent  casing,  a 
touch-screen  display  to  control  survey  parameters  and  acquisition  (right  inset),  and  a  DAQ  with  batteries 
mounted  on  a  backpack  frame  caried  by  the  second  operator  (left  inset).  The  MPV  can  use  two  positioning 
systems:  GPS  and  AHRS  for  open  field  survey;  a  beacon  boom  in  forest  or  steep  terrain. 
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The  MPV  is  a  handheld  sensor  with  wide-band,  time-domain,  EMI  technology.  The  sensor 
head  is  composed  of  a  single  transmitter  coil  and  an  array  of  five  receiver  units  that  measure  all 
three  components  of  the  EM  field  (Figure  1).  This  second-generation  MPV  is  specifically 
designed  to  (1)  be  man  portable  and  therefore  easy  to  deploy,  maneuver  and  adapt  to  a  survey 
environment,  and  (2)  acquire  data  that  is  suitable  for  discriminating  unexploded  ordnance  (UXO) 
from  non-UXO  targets.  The  MPV  head  is  a  50-centimenter  (cm)  diameter  transparent  disk.  The 
transmitter  coil  is  wound  around  the  disk  and  intennittently  illuminates  the  subsurface.  Five 
receiver  units  (cubes)  measure  the  three  orthogonal  components  of  the  transient  secondary  EM 
field  decay  with  three  air-induction  8-cm  square  coils  -  having  multiple  receivers  generally 
improves  the  recovery  of  target  parameters  for  classification  (Gasperikova  et  al.,  2007). 

The  MPV  is  a  programmable  instrument.  The  duration  of  the  excitation  and  time  decay 
recording  can  be  adjusted  to  accommodate  the  specific  needs  of  target  detection  and 
classification.  The  detection  survey  consists  of  a  full-coverage  sweep  where  dynamic  data  are 
collected  for  digital  geophysical  mapping  (DGM).  Fast  EMI  transmit-receive  cycles  are  applied 
so  that  the  sensor  can  continuously  move  (e.g.,  1  millisecond  [ms]  time  decay,  similar  to  Geonics 
EM-61).  The  quality  of  detection  data  may  not  always  be  sufficient  for  target  classification.  In 
such  case  a  target  is  reacquired  in  cued  interrogation,  where  data  quality  is  to  be  maximized.  The 
sensor  is  static  to  stack  the  recorded  signal  and  reduce  noise.  Longer  EMI  cycles  are  applied  to 
capture  variations  in  time  decay  rates  (e.g.,  25  milliseconds  (ms),  similar  to  Geonics  EM-63). 
This  late-time  information  has  been  shown  to  improve  distinction  between  intact  ordnance  and 
thinner  walled  shrapnel  and  cultural  debris  (Billings  et  al.,  2007). 


Figure  2:  The  MPV  detection  display  window  in  dynamic  data  collection  mode. 

The  top  left  panel  indicates  with  arrows  the  direction  of  the  nearest  compact  metallic  object  relative  to  the 
MPV  receiver  cubes  and  directs  the  operator  to  the  target  (here  the  MPV  sits  atop  the  target).  The  top  middle 
panel  shows  a  field  map  with  the  MPV  location  (red  dot)  and  azimuth  (black  line),  potential  target  locations 
(blue  dots),  and  cued  interrogation  soundings  (green  crosses).  The  middle  left  panel  provides  feedback  on 
positioning  sensors.  The  bottom  panel  has  the  data  acquisition  options  for  simple  one-button-touch  operation. 
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The  MPV  is  a  handheld  sensor.  The  sensor  head  weights  approximately  13  pounds  and  the 
backpack-mounted  data  acquisition  (DAQ)  and  batteries  weight  35  pounds.  Existing  sensors  with 
multiple  time  channel  measurement  capabilities  (e.g.,  Berkeley  UXO  Discriminator  [BUD], 
Geonics  EM63,  Time  Domain  EM  Towed  Array  Detection  System  [TEMTADS])  are  required  to 
be  mounted  on  a  cart  platfonn  due  to  the  size  and  weight  of  the  multiple  coils  of  wire  required 
for  the  transmitters  and  receivers. 

The  MPV  user  interface  has  real-time  data  monitoring  capabilities.  The  recorded  data  can  be 
displayed  to  verify  data  quality  and  detect  potential  disturbances  such  as  presence  of  magnetic 
soil  or  a  damaged  receiver.  The  past  and  present  sensor  location  is  displayed  on  a  map  along  with 
preset  survey  points  to  verify  spatial  coverage  and  global  location.  A  target  detection  and 
location  tool  indicates  the  origin  of  measured  EMI  fields  with  arrows  (the  so-called  “dancing 
arrows”  in  top  left  comer  of  Figure  2).  These  features  assist  the  field  operator  in  efficient  data 
collection,  so  that  detection  and  discrimination  data  can  be  collected  as  part  of  the  same  survey, 
thus  limiting  the  need  to  revisit  a  flag  for  further  characterization. 

2.1.1  Geolocation 

The  MPV  accommodates  different  positioning  systems  depending  on  site  conditions  and 
survey  needs.  For  detection  mapping  Global  Positioning  System  (GPS)  combined  with  Attitude 
and  Heading  Reference  System  (AHRS)  provide  navigation  and  positioning  in  open  field  or 
sparse  forest.  Under  dense  canopy,  sensor  location  can  be  estimated  with  a  cotton  thread  and 
spool  measuring  distance  along  track. 

For  inversion  and  classification,  target  parameters  are  best  recovered  when  a  target  is 
illuminated  from  multiple  directions  to  recover  its  transverse  polarizabilities.  When  deployed 
with  a  single  transmitter  loop  the  MPV  can  achieve  that  effect  by  acquiring  soundings  at  multiple 
locations  -  usually  five  -  around  the  anomaly.  This  technique  imposes  stringent  positioning 
constraints  for  reliable  classification  (Bell,  2005).  A  custom  positioning  system  was  designed  to 
meet  these  requirements  in  all  environments  without  the  line-of-sight  limitations  of  GPS  and 
roving  lasers.  The  “beacon”  system  (San  Filippo  et  al.,  2007)  consists  of  locating  the  origin  of 
the  primary  field  generated  by  the  active  MPV  transmitter  coil,  acting  as  a  beacon,  with  a  pair  of 
EMI  receivers  that  are  rigidly  attached  to  a  portable  beam  (Figure  1).  The  beam  is  placed  on  the 
ground  and  remains  at  the  same  location  during  cued  interrogation  of  an  anomaly,  like  a  base 
station,  to  derive  relative  sensor  location.  The  method  meets  accuracy  requirements  for  target 
classification  (Lhomine  et  al.,  2011)  with  1-2  cm  and  1-2  degrees  accuracy  for  position  and 
orientation  -  similar  to  GPS  -  out  to  distances  of  3-4  meters  (m).  The  beacon  has  been  utilized  at 
all  field  demonstrations  to  date. 

2.2  MPV  TECHNOLOGY  DEVELOPMENT 

The  project  was  initiated  in  2005  under  the  Strategic  Environmental  Research  and 
Development  Program  (SERDP)  MM-1443.  The  project  was  led  by  Drs.  Kevin  O’Neill  and 
Benjamin  Barrowes  with  the  Cold  Regions  Research  and  Engineering  Laboratory  of  the 
Engineering  Research  and  Development  Center  (CRREL,  ERDC)  in  Dartmouth,  New  Hampshire 
(NH).  The  first  MPV  prototype  was  built  in  2005-2006  with  David  George  of  G&G  Sciences, 
Grand  Junction,  Colorado  (CO).  It  was  tested  in  2007  at  ERDC  in  a  laboratory  setting.  Data 
analysis  showed  that  stable  target  parameters  could  be  retrieved  and  used  for  UXO  classification. 
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The  SERDP  project  was  first  extended  in  2008  to  continue  testing.  Field  trials  were  done  at  a 
test  plot  with  static  and  dynamic  acquisition  mode  over  buried  targets.  Stable  target  parameters 
were  recovered.  Effect  of  magnetic  soil  on  EMI  sensors  was  investigated.  Adverse  soil  effects 
could  be  defeated  owing  to  the  MPV’s  array  structure.  The  positioning  system  -  ArcSecond  laser 
ranger  -  proved  to  be  impractical  for  field  application  due  to  line-of-sight  requirement  for  all 
three  rovers  and  tedious  calibration.  The  SERDP  project  was  extended  to  test  an  alternative 
positioning  system  based  on  the  beacon  concept  and  prepare  modification  of  the  original  MPV 
prototype  for  extensive  field  deployments.  The  sensor  head  was  redesigned  with  lighter  materials 
and  a  smaller  head  diameter  to  reduce  weight  and  improve  maneuverability  while  maintaining  its 
expected  performance  (Lhomine,  2011b).  Receivers  were  placed  inside  transmitter  coil  to  reduce 
fragility;  transparent  material  was  employed  to  see  through  the  unit. 

Funding  was  obtained  in  2010  in  ESCTP  MR-201005  to  continue  developing  the  MPV  and 
perfonning  field  demonstrations.  The  MPV  fabrication  was  completed.  The  MPV  was 
successfully  demonstrated  at  YPG  UXO  test  site  in  October  2010  and  at  former  Camp  Beale  in 
June  2011.  High  detection  and  classification  rates  were  achieved. 

2.3  ADVANTAGES  AND  LIMITATIONS  OF  THE  MPV  TECHNOLOGY 

The  MPV  is  the  only  available  handheld  sensor  that  can  acquire  multi-static,  multi- 
component  data  on  a  wide  and  programmable  time  range.  The  MPV  offers  several  key  benefits: 

Hand-held  fonn  factor:  The  MPV  can  be  deployed  at  sites  where  terrain  and  vegetation 
preclude  use  of  heavier,  cart-based  systems.  Portability  can  improve  productivity  in  rough 
terrain.  The  system  is  easy  packable  and  transportable; 

Five  receivers  simultaneously  record  three  orthogonal  components  of  EM  field  with  near¬ 
perfect  relative  positioning  among  receivers.  Multi-component,  multi-axis  design  reduces 
number  of  soundings  for  target  characterization  (Barrowes  et  al.,  2007)  and  relaxes  positional 
accuracy  (Grzegorczyk  et  al.,  2009); 

Magnetic  soil  can  be  detected  and  defeated:  The  geometric  arrangement  of  receivers  and  the 
wide-band  time  range  offer  potential  for  identifying  and  neutralizing  the  effect  of  magnetic 
soil  through  techniques  developed  in  SERDP  MM- 14 14  and  MM- 1573. 

Fully  programmable  through  field  display:  Graphical  field-user  interface  controls  acquisition 
parameters  such  as  transmitter  wavefonn  characteristics,  duration  of  excitation,  number  of 
measurement  cycles,  stacking  and  recorded  time  channels. 

Highly  stable  EMI  components:  Responses  are  directly  predictable  using  standard  EMI 
theory.  Field  tests  verified  that  MPV  components  had  imperceptible  measurement  drift  and 
were  largely  insensitive  to  survey  conditions. 

Small  target  characterization:  Small  items  have  localized,  rapidly-varying  spatial  response. 
Voltage  in  an  air  induction  receiver  coil  is  an  average  of  a  target  scattered  field  through  the 
face  of  the  loop.  Therefore,  large  receivers  tend  to  “smear  out”  signals.  The  MPV’s  8  cm 
square  coils  are  typically  smaller  than  most  multi-channel  sensors  and  thus  better  suited  to 
detecting  and  sampling  signals  from  small  targets. 

Portability  has  limitations:  with  a  single  transmitter,  multiple  soundings  must  be  collected  to 
characterize  a  target.  Therefore  the  MPV  requires  (1)  an  accurate  positioning  system  for  cued 
interrogation  and  (2)  manual  intervention  to  move  the  sensor,  which  reduces  productivity  relative 
to  a  multi-transmitter  platfonn  for  which  a  single  sounding  is  often  sufficient. 
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3.0  PERFORMANCE  OBJECTIVES 

This  project  includes  data  collection  in  dynamic  detection  and  cued  interrogation,  data 
analysis  and  user  feedback  for  evaluation  of  the  MPV  technology.  The  specific  objectives  for 
each  stage  are  detailed  in  Table  1.  The  first  four  objectives  are  mostly  intrinsic  to  the  quality  of 
the  sensor  and  of  the  deployment  method.  The  data  were  analyzed  by  the  demonstrator  and  by 
other  ESTCP  partners. 


Table  1:  Performance  Objectives. 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Results 

Data  Collection  Objectives 

Spatial 
coverage  in 
detection 
survey 

Extended  footprint 
coverage 

•  Mapped  survey 
data 

95%  coverage 

95% 

Repeatability 
on  1VS 

Amplitude  of  EM 
anomaly 

Amplitude  of 
polarizabilities 

•  1VS  survey  data 

Detection 
amplitude  and 
inferred  size 
within  a  factor  3  of 
mean  value 

Detection: 
Amplitude  within 
factor  1.5 

Cued:  Size  within 
factor  1 . 1 

Detection  of  all 
targets  of 
interest 

Percent  detected  of 
seeded  TOl 

•  Location  of 
seeded  TOl 

•  Flag  list 

95%  within  0.3  m 
depth 

90%  from  0.3-1  m 
depth 

1 00%  of  seeds  were 
detectable 

3  seeds  missed 

Production  rate 

Daily  number  of 
cued  interrogations 
Pre-processing  time 

•  Log  of  field  work 
and  data  pre¬ 
processing  time 

100  flags 

Pre-proc.  time:  3 
min  per  target 

Mean:  125  per  day 
(Max  150) 

Ease  of  use 

•  Feedback  from 
operators/trainees 
on  usability  and 
training  time 

Quick  learning 
Detection: 
ergonomics  better 
for  tall  people 

Analysis  and  Classification  Objectives 

Maximize 

correct 

classification 

Number  of  TOl 
retained  and  rate  of 
clutter  rejected 

•  Ranked  dig  list 

•  Scoring  reports 
by  IDA 

Retain  at  least 

95%  TOl 

False  alarms 
reduction  by  at 
least  50% 

TOl  correctly 
classified: 

99%  (forest), 
100%(dynamic) 
85%  FAR  reduction 

Minimize 
number  of  flags 
that  cannot  be 
analyzed 

Number  of  “Cannot 
Analyze”  in 
classification 

•  Ranked  dig  list 

Reliable 

parameters  for  at 
least  95%  flags 

Over  99%  flags  with 
reliable  parameters 

Location  and 
depth  accuracy 
for  TOl 

Mean  error  and 
standard  deviation  in 
depth,  northing  and 
easting 

•  Location  of 
validated  within 
0.05  m 

•  Predicted 
location 

gAZ  <  0.10  m 
gAN  and  gAE  < 
0.30  m 

gAZ  =  0.05  m 
gAN  and  gAE  = 
0.20  m 
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3.1  OBJECTIVE:  SPATIAL  COVERAGE  FOR  DETECTION 

Dynamic  detection  survey  should  cover  a  maximum  of  the  area  of  interest  so  that  all 
detectable  targets  are  illuminated.  Targets  are  detectable  if  the  transmitted  field  is  sufficiently 
strong  to  reach  the  target  and  if  the  measured  target  response  is  sufficiently  strong  in  return  to 
exceed  a  given  threshold.  Simulations  suggest  that  there  is  negligible  loss  of  detect-ability  when 
a  target  is  located  10  cm  to  the  side  of  the  MPV  (Appendix  C).  We  can  therefore  assume  a  70  cm 
effective  diameter  footprint  for  the  MPV  sensor  head. 

3.1.1  Metric 

The  survey  footprint  is  compared  with  dynamic  survey  surface  area.  In  practice  the 
geographical  coordinates  of  MPV  receivers  are  binned  in  20-cm  square  cells.  The  ratio  of  the 
number  of  non-empty  cells  and  the  number  of  cells  in  the  survey  area  provides  the  rate  of 
coverage. 

3.1.2  Data  requirements 

The  geographic  coordinates  of  the  survey  perimeter  and  the  survey  track  are  utilized. 

3.1.3  Success  criterion  and  result 

The  objective  of  95%  spatial  coverage  was  attained. 

3.2  OBJECTIVE:  REPEATABILITY  OF  INSTRUMENT  VERIFICATION  TESTS 

Reliability  of  survey  data  depends  on  stability  of  survey  equipment.  The  IVS  was  tested 
twice  daily  on  the  Instrument  Verification  Strip  (IVS)  in  detection  mode  during  the  detection 
survey.  Cued  interrogation  of  IVS  targets  was  done  during  the  entire  demonstration. 

3.2.1  Metrics 

The  amplitude  of  the  MPV  data  and  that  of  the  target  parameters  span  multiple  orders  of 
magnitude.  For  instance,  the  raw  amplitude  of  a  target  response  doubles  for  every  5  cm 
increment  in  target  depth;  there  is  a  factor  10  between  the  amplitude  of  the  polarizabilities  of  a 
37  mm  and  a  75  mm  projectile,  and  their  time  decays  span  5  orders  of  magnitude  over  a  0.1-25 
ms  time  range.  Therefore  analyses  are  often  performed  on  the  logarithmic  values  of  these 
quantities.  The  metric  for  detection  is  the  amplitude  of  the  maximum  target  response,  defined  as 
the  nonn  of  the  total  field  on  a  cube.  The  metric  for  cued  interrogation  is  target  size,  here  defined 
as  the  norm  of  the  polarizability  components  at  early  time. 

3.2.2  Data  requirements 

The  IVS  survey  data  were  recorded  and  analyzed. 

3.2.3  Success  criterion  and  result 

This  objective  was  for  the  target  response  amplitude  and  the  size  factor  to  remain  within  a 
factor  3  of  the  mean  value.  In  practice  we  achieved  a  repeatability  within  a  factor  1.5  for 
detection  data  and  1.1  for  the  polarizabilities  inferred  from  cued  data  (Section  7.1). 
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3.3  OBJECTIVE:  DETECTION  OF  ALL  MUNITIONS  OF  INTEREST 

Target  detection  depends  on  signal  intensity,  spatial  coverage  and  the  target  picking  method. 

3.3.1  Metric 

The  metric  for  this  objective  is  the  percentage  of  flags  for  which  survey  data  exceed  the 
detection  threshold. 

3.3.2  Data  requirements 

Survey  data  were  recorded. 

3.3.3  Success  criterion  and  result 

The  objective  was  considered  to  be  met  if  detection  rate  exceeds  95%  within  0.3-m  depth  and 
90%  from  0.3-1  m  for  all  TOI  for  which  burial  depth  is  less  than  the  minimum  of  1  meter  and  10 
times  the  target  caliber.  All  targets  were  detectable.  However,  three  seeding  items  were  initially 
not  selected  because  two  laid  right  at  the  survey  boundary  and  one  target  was  merged  with 
another  pick  as  part  of  a  wide  anomaly.  A  root  cause  analysis  memorandum  was  submitted  to 
review  the  procedure  (included  in  Appendix  D). 

3.4  OBJECTIVE:  PRODUCTION  RATE 

Cued  interrogation  and  data  analysis  should  be  quicker  than  excavating  every  potential 
target.  Here  we  only  consider  data  collection  and  pre-processing.  The  latter  task  involves 
preparing  data  for  distribution  to  analysts  and  includes:  digitizing  field  notes,  consolidating  all 
soundings  per  flag,  inferring  sensor  position  with  the  beacon  and  merging  location,  removing 
background  and  nonnalizing  by  transmitter  current. 

3.4.1  Metric 

The  metrics  for  this  objective  are  the  mean  daily  survey  rate  and  the  mean  pre-processing 
time  per  flag. 

3.4.2  Data  requirements 

The  number  of  surveyed  flags  and  pre-processing  time  were  recorded  every  day. 

3.4.3  Success  criterion  and  result 

The  objective  of  100  flags  per  day  was  exceeded  with  a  mean  of  125  flags  and  two  days  with 
150  flags.  Originally  pre-processing  time  was  near  the  objective  of  3  minutes  per  flag.  However, 
we  later  found  occasional  drift  with  the  beacon  boom  compass  that  forced  a  re-analysis  of  the 
beacon  positions  and  increased  the  overall  time  to  5  minutes. 

3.5  OBJECTIVE:  EASE  OF  USE 

This  objective  is  qualitative.  One  new  field  personnel  was  involved  in  the  data  collection. 
The  crew  was  provided  with  initial  technical  material  for  familiarization  with  the  technology. 
The  mode  of  operation  was  explained  and  data  collection  was  explained  and  demonstrated  first 
on  the  IVS  and  training  pit  with  dynamic  and  cued  interrogation.  By  the  time  the  IVS  and  pit 
were  surveyed,  the  crew  was  able  to  collect  cued  data  over  simple  anomalies  and  recognize  more 
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complex  situations,  e.g.,  offset  targets,  and  ask  for  confirmation  of  the  appropriate  task  to 
perfonn.  Overall  the  experienced  field  technician  and  the  trainee  collected  equal  amount  of  data 
with  similar  quality  and  productivity.  Overall  data  quality  was  satisfactory  -  spatial  coverage  met 
the  objective,  few  recollects  -  which  suggests  a  successful  technology  transfer. 

In  terms  of  ergonomics  we  found  that  the  sensor  handle  was  better  designed  for  tall  people, 
who  were  also  at  greater  ease  for  sweeping  the  sensor  head  across  track  in  detection  survey. 

3.6  OBJECTIVE:  MAXIMIZE  CORRECT  CLASSIFICATION 

This  is  the  most  important  objective  of  the  demonstration:  all  shallow  munitions  should  be 
recommended  for  excavation  after  geophysical  inversion  and  a  maximum  of  clutter  should  be 
dismissed.  This  objective  requires  that  sufficient,  high-quality  data  are  collected,  and  that  the 
ensuing  data  analysis,  inversion  and  statistical  classification  process  recognizes  the  presence  of 
munitions.  The  prioritized  dig  list  indicates  an  estimate  of  target  type  and  a  stop  dig  point. 

3.6.1  Metric 

The  metric  for  this  objective  is  the  rate  of  TOI  that  are  recognized  as  such  and  the  rate  of 
clutter  left  after  the  stop  dig  point. 

3.6.2  Data  requirements 

Ranked  dig  lists  were  submitted  to  the  ESTCP  Program  Office  for  scoring  by  the  Institute  for 
Defense  Analyses  (IDA). 

3.6.3  Success  criterion  and  result 

The  objective  was  to  identify  at  least  95%  of  the  UXO  with  at  least  50%  clutter  rejection.  For 
the  forested  area  we  achieved  99%  correct  classification  with  85%  clutter  rejection,  missing  one 
ISO  in  a  fragment  pit  among  four  other  pieces.  The  dynamic  area  was  processed  in  two  stages: 
approximately  half  of  the  anomalies  were  successfully  classified  using  dynamic  data,  while  the 
remaining  ones  were  classified  using  cued  data.  We  found  100%  of  the  TOI  and  rejected  over 
85%  of  the  clutter  (details  in  Section  7.2). 

3.7  OBJECTIVE:  MINIMUM  NUMBER  OF  FLAGS  THAT  CANNOT  BE  ANALYZED 

Some  flags  may  not  be  classified  due  to  insufficiently  informative  data  or  inadequate  data 
processing.  The  former  is  a  measure  of  instrument  perfonnance  and  field  practices;  the  latter  is  a 
measure  of  data  analysis  quality. 

3.7.1  Metric 

The  metric  for  this  objective  is  the  number  of  flags  that  cannot  be  analyzed  by  our  method, 
and  the  intersection  of  all  dig  lists  among  all  analysts. 

3.7.2  Data  requirements 

Each  analyst  submitted  their  dig  list  for  scoring  by  IDA. 
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3.7.3  Success  criterion  and  result  evaluation  and  results 

The  objective  to  classify  at  least  95%  of  the  flags  with  the  MPV  depth  range  of  investigation 
was  exceeded  with  over  99%.  Only  three  flags  could  not  be  classified  because  their 
corresponding  anomalies  were  offset  by  over  50  cm  and  the  field  operator  did  not  extend  the 
survey  pattern  to  fully  cover  the  anomaly  (Details  in  Appendix  F.l).  These  flags  were  failed, 
though  technically  they  could  have  been  classified  as  empty  holes. 

3.8  OBJECTIVE:  LOCATION  AND  DEPTH  ACCURACY  FOR  TOI 

Correct  target  classification  relies  on  the  capability  to  extract  accurate  target  parameters. 
Accurately  locating  potential  TOI  is  important  for  safe  and  efficient  site  remediation. 

3.8.1  Metric 

The  metric  is  the  difference  between  observed  and  predicted  depth  and  geographic  location. 

3.8.2  Data  requirements 

Predicted  location  and  depth  are  recorded  among  target  parameters  and  compared  to  ground- 
truth  validation  measurements. 

This  objective  requires  accurate  ground  truth.  This  condition  may  be  challenging  given  that 
flags  may  have  been  removed  between  the  interrogation  and  excavation  events,  and  flags  might 
have  also  been  misplaced  and  not  lay  sufficiently  close  to  the  anomaly  of  interest. 

3.8.3  Success  criterion  and  result 

Depth  should  generally  be  predicted  within  0.10  m  and  geographic  location  within  0.30  m 
(the  typical  radius  of  an  excavation  hole).  The  standard  deviation  for  the  difference  between  the 
predicted  and  observed  depth  and  location  are  0.05  m  and  0.20  m,  respectively,  and  the  means 
are  close  to  zero. 
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4.0  SITE  DESCRIPTION 


The  site  description  material  reproduced  in  this  section  is  taken  from  the  ESTCP 
Demonstration  Plan  for  Spencer  Range  (Draft  3),  where  reference  to  additional  material  can  be 
found.  The  former  Spencer  Artillery  Range  is  a  30,618  acre  site  located  near  Spencer,  Tennessee. 
The  demonstration  was  conducted  in  a  portion  of  the  Munitions  Response  Site  (MRS)  1.  An 
aerial  photo  of  the  demonstration  area  is  shown  in  Figure  3. 
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Figure  3:  Aerial  photograph  of  demonstration  site. 

In  the  upper  right  panel,  orange  contour  indicates  the  forest  survey  area  (portable  systems  only),  green  line 
limits  dynamic  area  where  all  sensors  were  deployed  in  dynamic  and  static  mode  (except  BUD)  and  while 
blue  area  was  set  for  the  MetalMappers  only. 


4.1  SITE  SELECTION 

This  site  was  chosen  as  the  next  in  a  series  of  sites  for  demonstration  of  the  classification 
Process.  The  first  site  in  the  series,  former  Camp  Sibert,  had  only  one  target-of-interest  and  item 
“size”  was  an  effective  discriminant.  A  hillside  range  at  the  former  Camp  San  Luis  Obispo  in 
California  was  selected  for  the  second  of  these  demonstrations  because  of  the  wider  mix  of 
munitions,  including  60-mm,  81 -mm,  and  4.2-in  mortars  and  2.36-in  rockets.  Three  additional 
munitions  types  were  discovered  during  the  course  of  the  demonstration.  The  third  site  chosen 
was  the  former  Camp  Butner  in  North  Carolina.  This  site  is  known  to  be  contaminated  with 
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items  as  small  as  37-mm  projectiles,  adding  yet  another  layer  of  complexity  into  the  process. 
Additional  sites  including  this  one  provide  opportunities  to  demonstrate  the  capabilities  and 
limitations  of  the  classification  process  on  a  variety  of  site  conditions.  This  site  was  selected  for 
demonstration  because  it  is  more  heavily  wooded  than  prior  demonstrations  and  is  thought  to 
contain  a  wide  mixture  of  munitions.  These  two  features  increase  the  site’s  complexity  and  both 
characteristics  are  likely  to  be  encountered  on  production  sites.  The  tree  cover  poses  a  navigation 
challenge  by  increasing  the  difficulty  of  obtaining  accurate  global  positioning  system  (GPS) 
readings. 

4.2  SITE  HISTORY 

In  1941,  construction  began  on  the  30,618  acre  Spencer  Artillery  Range  and  documentation 
identifies  establishment  of  two  impact  areas:  Jakes  Mountain  (5,060  acres)  and  Bald  Knob 
(2,090  acres).  Troop  training  took  place  until  September  1944,  by  which  time  Army  ground 
forces  had  either  departed  or  were  under  orders  to  depart.  Subsequent  arrangements  were  made 
for  Dyersburg  Anny  Air  Field  to  use  the  Spencer  Artillery  Range  as  an  air-to-ground  gunnery 
range.  The  land  reverted  back  to  the  original  25  leaseholders  in  the  summer  of  1946.  Several 
surface  decontamination  sweeps  were  completed  on  portions  of  the  former  range  in  the  1950s. 
Since  then,  numerous  tracts  of  land  have  been  sold  and/or  subdivided,  significantly  increasing 
the  number  of  property  owners  from  the  original  25  to  several  hundred  landowners  today. 

4.3  MUNITIONS  CONTAMINATION 

The  suspected  munitions  at  this  site  include: 

•  37mm  projectiles, 

•  75mm  projectiles, 

•  76mm  projectiles, 

•  105 mm  proj  ectiles ,  and 

•  155mm  projectiles. 

Small  Industry  Standard  Objects  (IS02)  were  added  to  increase  the  number  of  native  munitions. 

4.4  SITE  CONFIGURATION 

The  boundaries  of  the  three  sub-areas  are  shown  in  Figure  3.  Note  that  the  open  field  area 
had  been  recently  cleared  of  trees,  but  that  is  not  reflected  in  the  aerial  photo.  Shape  files 
delineating  the  demonstration  site  boundaries  and  the  underlying  grid  system  were  made 
available  from  the  ESTCP  Program  Office. 
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5.0  TEST  DESIGN 


The  goal  of  the  study  is  to  demonstrate  and  characterize  detection  and  discrimination  with  the 
MPV.  Sensor  classification  performance  is  characterized  as  a  function  of  the  size  and  depth  of 
the  buried  targets  and  the  presence  and  effect  of  aggravating  factors  (nearby  object,  magnetic  soil 
and  complex  terrain). 

5.1  DEMONSTRATION  SCHEDULE 

The  field  survey  was  completed  in  two  weeks.  The  daily  report  is  included  in  Appendix  E. 
The  first  day  was  set  for  preparing  the  survey:  unpack,  assemble  the  sensor,  set  up  the  survey, 
train  crew  and  acquire  calibration  measurements  on  training  pit  and  IVS.  Because  of  the 
presence  of  another  field  crew  the  survey  began  in  the  forested  area  before  proceeding  to  the 
dynamic  detection  area,  which  lasted  just  over  2  days.  The  dynamic  survey  data  were  analyzed 
by  the  PI  while  the  crew  moved  to  cued  data.  Most  of  the  data  pre-processing  was  performed 
during  the  deployment.  Feature  extraction  and  classification  was  done  in  the  following  months. 
The  Gantt  chart  in  Table  2  shows  the  schedule  for  each  phase  of  testing  and  how  the  various 
phases  are  related. 


Table  2:  Demonstration  steps 


Preparation 

Calibration 

Field  surveys 

Post  survey 
analysis 

Tasks  and  demonstration  timeline 

1.1 

1.2 

1.3 

2.1 

2.2 

2.3 

3.1 

3.2 

3.3 

Mobilization  -  Demobilization 

X 

X 

CAEIB RATION:  Training  pit 

X 

CAEIB RATION:  Inversion  and  parameters 
stability  analysis  of  training  pit  data 

X 

CAEIB  RATION:  Twice-daily  test  strip  survey 
and  data  analysis 

X 

X 

X 

DYNAMIC  SURVEY :  Detection  in  dynamic 
area 

X 

DYNAMIC  SURVEY:  Target  picking 

X 

CUED  SURVEY:  Wooded  area  and  dynamic 
area 

X 

DATA  PRE-PROCESSING:  Cued  data 

X 

X 

DATA  ANALYSIS:  feature  extraction 

X 

DATA  ANALYSIS:  classification  ranked  list 

X 

RETROSPECTIVE  ANALYSIS 

X 

REPORTING 

X 

5.2  SYSTEM  SPECIFICATION 

For  cued  interrogation  mode  the  system  is  set  for  25  milliseconds  (ms)  excitation  and  25  ms 
recording  of  EMI  transients.  This  is  accomplished  by  using  0.9  seconds  (s)  data  blocks  that 
include  9  repeats  (100  ms  per  cycle).  Station  time  is  set  to  6.3  s  by  stacking  7  data  blocks 
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(effectively  9  x  7  =  63  cycles  are  averaged).  Digital  receivers  use  a  4  microsecond  sampling  rate. 
The  data  are  recorded  with  133  logarithm-spaced  time  gates  (0.05%  gate  width)  from  0-25  ms. 
Dynamic  survey  is  set  with  2.7  ms  time  window  and  short  0.1 -s  data  block  so  as  to  reduce 
smearing  of  the  signal  by  sensor  motion. 

The  dynamic  area  has  open  sky  and  positioning  is  based  on  the  GPS.  In  cued  mode  local 
positioning  is  achieved  with  the  beacon  system,  though  the  GPS  data  are  still  recorded  to  verify 
beacon  accuracy  whenever  enough  satellites  are  visible,  in  particular  at  the  IVS  and  in  the  open- 
field  area.  The  GPS  is  a  Trimble  R8  that  is  mounted  on  the  opposite  end  of  the  MPV  handling 
boom.  The  GPS  is  also  used  to  locate  pre-programmed  flag  locations.  A  XSens  MTi  orientation 
sensor  is  mounted  near  the  GPS  to  measure  azimuth.  The  three-axis  sensor  data  is  also  used  for 
verifying  the  pitch  and  roll  inferred  from  the  beacon  measurements.  The  beacon  boom  is  laid  on 
the  ground  within  2  meters  of  the  survey  flag.  Boom  orientation  is  recorded  with  a  secondary 
XSens  orientation  sensor  that  is  mounted  at  the  boom  center.  Boom  is  generally  oriented  in  the 
North-South  direction  on  left  side  of  target.  After  data  processing,  beacon-derived  positions  are 
located  relative  to  the  local  flag  and  the  geographic  North,  and  subsequently  globally-referenced 
using  the  supplied  GPS  coordinates  of  each  flag. 

5.3  CALIBRATION  ACTIVITIES 

Calibration  is  designed  to  verify  correct  sensor  operation  and  calibrate  the  recorded  sensor 
response  over  known  targets.  It  was  expected  that  37mm,  75  mm  and  105  mm  projectile  and 
small  ISO  could  be  encountered  at  the  site.  A  target  sample  set  was  available  for  test  pit 
measurements.  Each  item  was  successively  placed  inside  a  clutter-free  training  pit  and  surveyed 
in  cued  interrogation  mode  (Figure  4).  A  minimum  of  four  different  orientations  and  one  depth 
per  target  were  tested  so  that  feature  extraction  and  classification  methods  could  be  trained. 
Combinations  with  two  objects  were  also  tested. 


Figure  4:  Data  collection  over  test  pit.  Reference  TOI  were  placed  in  empty  hole  and  surveyed  in  cued  mode. 
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The  sensor  was  checked  by  surveying  twice  a  day  over  an  Instrument  Verification  Strip 
(IVS)  where  known  targets  are  buried  in  a  clutter-free  environment.  The  strip  was  surveyed  in 
dynamic  and  in  cued  modes  so  that  detection  and  classification  could  be  verified.  Data  were 
immediately  inverted  as  an  indirect  means  to  verifying  sensor  reliability  by  examining  stability 
of  the  recovered  target  parameters. 

In-air  measurements  were  regularly  acquired,  in  particular  before  and  after  any  battery 
change,  to  monitor  variations  in  transmitter  power  and  instrument  noise.  Background 
measurements  were  acquired  with  the  sensor  head  on  the  ground  for  every  tenth  flag,  after 
identifying  a  nearby  “quiet”  area  that  was  diagnosed  by  examining  the  recorded  decay  curves  in 
static  mode.  Data  were  subsequently  analyzed  to  quantify  the  spatial  and  temporal  variability  in 
background  noise  and  detect  potential  soil  viscous  magnetization.  We  found  that  the  background 
signal  was  significantly  weaker  than  that  of  the  Camp  Beale  study  (Figure  5). 


Rx  X  Rx  Y  Rx  Z 


Time  (ms)  Time  (ms)  Time  (ms) 


Figure  5:  Background  signal  for  static  data  at  Spencer  Range  (field  and  IVS). 

The  median  value  is  shown  for  each  of  the  15  receivers  (X,Y  and  Z  components  and  cubes  1-5)  and  it  is 
compared  between  the  IVS,  the  forest  and  Camp  Beale.  One  third  of  the  cued  interrogations  were  done  with  a 
ground  clearance  of  5  cm,  using  a  foam  pad  riser;  the  corresponding  background  is  shown  for  the  IVS. 

Beacon  positioning  was  verified  by  comparing  the  locations  that  were  predicted  with  the 
GPS/AHRS  and  beacon  positioning  systems  wherever  some  of  the  GPS  signal  was  usable. 
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5.4  DATA  COLLECTION  PROCEDURES 


5.4.1  Sample  density 

Similar  to  previous  demonstrations,  cued  interrogation  soundings  were  collected  around  the 
marked  target  location  (ground  paint  or  flag).  The  first  sounding  was  acquired  at  the  marker.  In 
general  the  operator  followed  a  standard  five-point  square  pattern  as  in  Figure  6.  At  Camp  Beale 
the  pattern  was  conservatively  augmented  to  9  points  but  retrospective  analysis  showed  that  the 
additional  soundings  brought  negligible  benefit  to  classification.  With  five  sounding  and  a  cube 
separation  of  0.2  m,  a  somewhat  uniform  spatial  sampling  is  obtained  with  0.6-0. 7  m  spacing 
between  soundings  locations. 


Figure  6:  Cued  survey  pattern  with  five  points  centered  on  marked  target  location. 

Detection  survey  was  performed  by  walking  along  pre-defined  survey  lines  (Figure  7)  that 
were  pre-programmed  in  the  data  acquisition  software  and  physically  laid  on  the  ground.  The 
sensor  head  was  swept  from  side  to  side  while  keeping  the  sensor  head  parallel  to  the  ground  - 
the  sensor  track  resembles  a  figure  of  “S”.  Given  a  sensor  footprint  of  0.7  m,  we  started  with  a 
1.5-m  line  spacing  but  found  that  the  sweeping  amplitude  was  too  wide  for  our  shortest  field 
operator.  Most  of  the  survey  was  done  at  1.2  m  spacing.  Station  spacing  depends  on  survey 
speed.  Following  an  empiric  rule  such  that  the  sensor  should  not  move  more  than  the  receiver 
length  (8  cm)  during  acquisition  of  a  data  block  (0.1  s),  sensor-head  speed  should  be  between 
0.5-1  m/s  with  station  spacing  of  0.05-0.1  m  along  each  of  the  5  receiver-cube  tracks.  The 
resulting  along-line  speed  is  approximately  0.3  m/s. 


Figure  7:  Picture  of  dynamic  survey  with  MPV. 
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5.4.1  Quality  checks 

For  detection  survey  the  sensor  track  is  displayed  on  the  field  monitor.  The  operator  regularly 
checked  the  monitor  for  possible  gaps.  The  second  operator  watched  the  main  operator  to  verify 
the  survey  speed,  sweeping  amplitude  and  obvious  gaps.  For  cued  interrogation,  each  sounding  is 
displayed  immediately  after  acquisition  to  help  the  operator  verify  adequate  anomaly  sampling 
and  data  quality  by  examining  data  decay  curves  (Figure  8)  and  the  arrows  display.  The  first 
sounding  requires  particular  attention  to  verify  that  the  signal  source  originates  right  below  the 
marked  location.  Offset  can  result  from  positional  errors  or  difference  in  sensitivity  between  the 
detection  sensor  and  the  MPV  and  caused  by  an  offset  target  or  multiple  targets.  In  such  cases 
the  operator  is  expected  to  apply  close  scrutiny  to  interpret  all  soundings,  locate  if  possible  signal 
source  and  acquire  additional  soundings  if  necessary.  Anomaly  coverage  is  verified  by  ensuring 
that  the  furthest  receiver  is  measuring  background.  If  residual  signal  from  the  target  remains  then 
additional  soundings  are  collected  to  ensure  full  coverage  of  the  anomaly  spatial  decay.  For 
instance,  if  the  MPV  front  receivers  show  above-background  signal  when  the  MPV  is  placed  in 
position  2  (Figure  6),  then  a  sounding  is  to  be  collected  North  of  the  middle  of  positions  2-3.  If  a 
nearby,  interfering  target  is  detected  while  being  un-flagged  for  cued  interrogation,  then 
supplementary  soundings  are  acquired  to  improve  characterization  of  the  two  sources. 

Data  quality  checks  are  first  done  during  the  survey  to  verify  that  receivers  are  properly 
operating.  Any  abnormal  sounding  is  tagged  and  a  new  sounding  is  acquired  at  same  location.  In 
case  of  receiver  failure  survey  is  stopped  until  a  solution  is  found  -  this  never  happened.  Data 
quality  is  also  controlled  post-survey,  while  still  on  site,  to  identify  possible  issues  and  need  for 
re-acquisition.  Particular  features  to  monitor  are  the  spatial  coverage  of  the  anomalies,  especially 
if  the  flag  and  anomaly  peak  are  offset,  beacon  positions  and  signal  to  noise  ratio. 


Figure  8:  Typical  target  response  when  the  MPV  head  is  placed  directly  above  a  buried  target. 

The  Z-component  data  show  that  target  is  closest  to  center  cube  (#3)  and  equally  distant  from  lateral  cubes  2 
and  4,  while  signal  in  cube  5  resembles  background.  The  Y  data  confirm  that  target  is  buried  between  front 
and  back  cubes  (1,  5)  and  X  data  confirm  that  target  is  located  between  side  cubes  2  and  4. 


We  ensure  that  all  flags  are  visited  by  pre-programming  their  GPS  coordinates  and 
displaying  their  location  on  the  sensor  display  map  (Figure  2).  Despite  occasional  GPS  signal 
drops  under  the  tree  canopy,  the  GPS  generally  helps  navigating  between  flags.  Each  visited  flag 
is  automatically  marked  on  the  map. 
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5.4.2  Data  handling 

Data  are  stored  as  .tem  files  on  the  DAQ  and  converted  to  .csv  files  before  every  battery 
change.  We  keep  a  copy  of  all  .tem  and  .csv  files  on  the  DAQ,  on  a  portable  hard-disk  drive  and 
on  the  field  laptops  that  is  used  for  reviewing  the  data.  The  backpack  bearing  operator  documents 
the  survey  by  noting  target  names  and  file  numbers  in  addition  to  any  remarks  made  by  the 
principal  operator.  Field  notes  are  digitized  every  day  by  taking  pictures  of  the  notes  and  filling 
out  a  spreadsheet  that  is  used  for  pre-processing. 
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6.0  DATA  ANALYSIS  PLAN 

6.1  PREPROCESSING 

The  MPV  computer  DAQ  records  data  streams  from  the  sensor  head,  beacon  receivers, 
attitude  sensor  and  GPS  with  the  GPS  time  stamp.  The  DAQ  saves  the  data  into  a  .tern  binary 
file.  Each  sounding  data  is  converted  to  a  .csv  file  without  any  data  alteration.  Several  pre¬ 
processing  stages  are  perfonned  before  delivery  to  the  analysts. 

The  beacon  receiver  data,  transmitter  current  and  attitude  measurement  are  combined  to  infer 
the  MPV  head  location.  When  the  GPS  Q  factor  is  equal  to  4  the  GPS  and  attitude  can  be 
combined  to  predict  the  sensor  head  location  and  compare  to  the  beacon.  In  case  of  discrepancy 
we  consider  the  field  notes,  the  relative  distance  to  the  beacon  boom  and  any  possible  disrupting 
factors  to  choose  the  most  likely  solution  or  discard  the  sounding.  At  Camp  Beale  the  beacon 
position  was  valid  100%  of  the  time. 

The  MPV  head  EMI  data  is  divided  by  the  recorded  transmitter  current  amplitude  at  turn  off 
to  nonnalize  the  response  to  a  unit  transmitter  excitation,  hence  compensating  for  fluctuations  in 
transmitter  battery  power.  Background  measurements  are  analyzed  to  define  a  background 
response  to  be  subtracted  from  the  cued  data.  A  spatial  parameterization  of  the  background 
response  can  be  utilized  if  significant  spatial  variability  is  observed.  The  resulting  data  is  visually 
validated. 

Because  each  sounding  generates  an  individual  data  file,  files  must  be  combined  to  a  single 
record  for  each  target.  The  merged  file  is  composed  of  data  blocks  for  each  sounding  with  the 
sensor  location  and  attitude,  sounding  number  and  field  comment.  Only  validated  soundings  are 
included.  The  final  file  name  is  comprised  of  the  sensor  and  target  names  following  ESTCP 
naming  instructions.  Files  are  later  posted  on  a  ftp  site  for  distribution  to  analysts. 

6.2  TARGET  SELECTION  FOR  DETECTION 

Dynamic  detection  surveys  are  a  novelty  to  this  project.  Dynamic  data  were  collected  at  YPG 
and  detection  thresholds  were  identified.  However,  the  site  was  clutter-free  and  only  contained 
single  targets  at  fixed  locations.  Therefore  that  data  provided  limited  in-sight  into  live-site 
conditions  and  the  target  picking  process  was  finalized  on  site  based  on  prior  simulations  and 
comparison  with  local  conditions  and  noise  levels.  Technical  material  on  target  detection  are 
included  in  Appendix  C  and  D. 

6.3  PARAMETER  ESTIMATION 

Advanced  data  analysis  is  done  with  UXOLab,  a  MatLab-based  software  jointly  developed 
with  the  University  of  British  Columbia  and  used  in  numerous  ESTCP  and  SERDP  projects. 
Data  are  inverted  using  a  three-dipole  instantaneous  polarization  model  (Pasion  and  Oldenburg, 
2001).  The  target  polarizability  decay  parameters  are  the  main  features  for  the  ensuing 
classification.  Inversion  setup  parameters  such  as  noise  estimation  are  generally  decided  upon 
examination  of  the  training  pit  data.  Solutions  with  one  or  multiple  targets  are  generated  for 
every  selected  target.  Decisions  regarding  the  number  of  targets  at  a  given  location  are  made 
through  statistical  classification  by  prioritizing  the  most  munitions-like  solutions.  Inversion 
results  are  reviewed  by  an  experienced  geophysicist  to  identify  any  potential  issues  with  the 
inversion  setup  or  with  the  data,  and  select  data  subsets  as  required  for  fitting  all  detected 
anomalies  (masking). 
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6.4  TRAINING 

Statistical  classifiers  were  trained  on  a  library  of  target  features  that  had  been  accumulated 
during  the  previous  surveys  at  YPG  and  Camp  Beale,  and  adding  new  features  associated  with 
local  targets.  Measurements  were  collected  over  the  training  pit  with  local  munitions.  The  items 
were  placed  in  various  positions  and  orientations  so  that  parameter  variance  could  be  estimated. 

After  testing  of  the  classifier,  additional  training  data  were  requested  to  the  ESTCP  to  obtain 
information  about  particular  targets.  Targets  were  chosen  because  they  belonged  to  a  cluster  of 
unknown  targets  with  similar  features,  or  because  some  particular  features  stood  out.  Training 
data  were  iteratively  requested  until  sufficient  confidence  in  the  classifier  was  attained.  The 
classification  decision  memorandum  detailed  the  training  selection  process  (Appendix  F). 

6.5  CLASSIFICATION 

The  following  guiding  principles  were  applied: 

Selection  of  features:  By  analysis  of  the  training  data,  those  features  that  contribute  to 
separation  of  the  different  classes  (comprising  UXO  types  and  clutter)  are  selected.  Our 
experience  shows  that  the  three  sets  of  instant  polarizability  decays  generally  yield  successful 
classification  with  the  MPV  (and  other  sensor  data).  The  data  are  inverted  in  different 
manners,  using  single-target  and  multiple-target  inversions  and  eventually  different  noise 
parameters  or  mask  sizes.  Therefore  multiple  sets  of  features  can  be  extracted  from  the  same 
anomaly  and  the  model  that  most  likely  resemble  a  TOI  is  automatically  selected  through 
classification; 

Choice  of  classification  algorithm:  Methods  are  elaborated  through  analysis  of  the  training 
data.  Support  Vector  Machine  (SVM)  has  been  efficient  at  the  past  demonstration  when 
applied  through  a  cascading  algorithm.  Library  misfit  has  also  shown  great  success  and  has 
the  benefit  of  being  intuitively  easier  to  comprehend; 

Classification:  Anomaly  labels  are  placed  in  a  prioritized  dig-list  by  using  the  classifier  to 
compute  probabilities  of  class  membership  for  unlabeled  feature  vectors.  The  most  likely 
TOI  is  reported  in  the  dig  sheet; 

Number  of  UXO-classes :  When  using  a  statistical  classifier  such  as  SVM,  multiple  UXO 
classes  can  be  used  to  account  for  difference  in  sizes.  For  library  misfit,  each  reference  item 
forms  its  own  class;  there  can  be  multiple  instances  of  a  given  target  type  to  account  for  its 
observed  parameter  variability,  in  particular  for  large  objects  for  which  orientation  may 
affect  the  recovered  polarizabilties. 

We  followed  a  similar  method  to  that  developed  and  successfully  demonstrated  in  the  past 
ESTCP  demonstrations.  Because  the  effect  of  magnetic  soil  was  noticeably  weaker  than  at  Camp 
Beale,  we  were  able  to  apply  a  standard  classification  methods  based  on  library  misfit.  The 
method  was  detailed  in  the  classification  memorandum  (included  in  Appendix  F).  The  TOI 
library  was  initially  based  on  items  found  at  previous  sites.  The  library  was  modified  after 
training,  removing  items  that  did  not  seem  to  be  present,  and  adding  TOI  which  features  differed 
from  their  typical  reference  class. 

A  multi-stage  classifier  was  applied.  The  first  stage  included  all  polarizabilities  (L123)  and 
the  next  stages  only  used  a  combination  of  two  polarizabilities  (L12,  L13),  and  the  final  stage 
only  the  main  one  (LI).  Targets  for  which  training  data  were  available  were  also  included  in  the 
classifier  to  assess  the  effect  of  classification  parameters  on  their  ranking.  The  decision  to  switch 
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from  one  classifier  stage  to  the  next  was  based  on  a  decision  statistic  derived  from  the  library 
misfit  metric.  For  each  stage,  that  metric  shows  a  strong  inflection  when  items  strongly  differ 
from  library  TOI  (Figure  36  in  Appendix  F).  The  stage  switch  (or  stop  digging  within  this 
classifier  stage)  was  automatically  computed  and  further  confirmed  by  visual  inspection  of  the 
polarizabilities  in  ranked  order  (Figure  9)  and  comparison  with  ground  truth  data  (obtained  from 
training  or  earlier  dig  list  stages). 


Figure  9:  Polarizabilities  for  ranked  dig  list  at  classification  stage  2. 

In  each  frame  the  target  polarizabilities  are  compared  to  their  nearest  library  item,  which  caliber  is  indicated 
in  a  small  text  box.  Target  information  is  indicated  with  target  identity  and  misfit  in  format  Tnnnn/misfit. 
The  text  box  is  highlighted  in  yellow  for  a  validated  TOI  and  in  pink  for  clutter.  Blue  boxes  indicate  stage 
switches  in  classifier. 

The  end  product  of  classification  is  a  ranked  dig  list  that  was  formatted  as  in  Figure  10.  The 
first  items  on  each  list  were  those  anomalies  for  which  reliable  parameters  could  not  be  extracted 
and  therefore  required  excavation.  Next  followed  the  training  items,  then  the  “high  confidence’ 
munitions.  Items  were  ranked  according  to  decreasing  confidence  that  the  item  was  hazardous. 
Any  items  that  were  analyzed  without  reaching  an  unambiguous  classification  decision  were 
placed  next  on  the  dig  list.  Finally,  all  items  that  were  confidently  classified  as  non-hazardous 
were  ranked  by  their  confidence. 
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Classification  was  first  applied  to  the  cued  data  that  were  collected  in  the  forested  area, 
following  the  method  presented  here-above.  Study  of  the  dynamic  area  followed  a  slightly 
different  approach.  Classification  was  performed  in  two  steps,  first  using  inverted  dynamic  data 
wherever  results  seemed  reliable,  like  a  pre-screener,  and  using  cued  data  only  for  the  remaining 
flags.  This  being  the  first  time  the  method  was  applied,  a  conservative  approach  to  selecting 
acceptable  results  for  dynamic  data  classification  was  adopted.  Only  models  with  high  quality 
data  fits  and  high  spatial  coverage  were  kept.  This  resulted  in  retaining  only  50%  of  the  flags  for 
the  dynamic  classification  stage.  We  did  not  request  training  data  for  the  dynamic  area,  relying 
only  on  the  dynamic  I  VS  data  and  the  training  data  from  the  forested  area.  The  study  was 
completed  in  two  stages  and  no  TOI  was  missed  (Section  7.2). 


Initial  Ranked  Anomaly  List 


Final  Ranked  Anomaly  List 


Anomaly 

ID 

Category 

Dig  on 
First  Pass 

Type 

Comment 

2498 

-1 

1 

Training  Data 

247 

-1 

1 

1114 

0 

1 

Can't  extract  reliable  parameters 

883 

2 

1 

1642 

2 

1 

Unable  to  classify 

713 

2 

0 

406 

3 

0 

3 

0 

High  likelihood  not  TOI 

3 

0 

3 

0 

3 

0 

3 

0 

3 

0 

First 

Pass 

Threshold 


Figure  10:  Format  of  prioritized  dig  list  to  be  submitted  to  ESTCP  Program  Office. 
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7.0  PERFORMANCE  ASSESSMENT 

7.1  IVS  repeatability 

The  IVS  was  surveyed  in  detection  and  cued  interrogation  mode.  Data  were  inverted  to 
indirectly  assess  repeatability.  Inverted  polarizabilities  for  the  cued  data  are  presented  in  Figure 
1 1 .  The  polarizabilities  are  almost  perfectly  equal,  which  confirms  that  the  sensor  was  properly 
operating  and  that  the  operator  was  following  a  repeatable  process  that  allowed  for  sufficient 
characterization  of  the  buried  targets. 


Figure  11:  Polarizabilities  on  IVS  lane:  (A)  75  mm,  (B)  37  mm,  (C)  Shotput  and  (D)  IS02. 


Dynamic  data 

The  dynamic  data  were  analyzed  to  verify  detection  of  the  IVS  items.  Data  were  also 
inverted  to  provide  features  for  classification  and  assess  stability.  A  detection  map  of  5  passes  is 
presented  in  Figure  12,  which  confirms  that  all  anomalies  are  detected  with  similar  amplitude 
and  footprint  size  on  each  pass.  The  first  pass  had  some  positional  issues  as  the  GPS  update  rate 
was  too  low,  leaving  large  gaps  between  points  -  this  is  most  visible  on  the  second  target  from 
the  top.  This  was  addressed  before  starting  the  field  survey. 

The  inverted  polarizabilities  are  presented  in  Figure  13.  Besides  the  first  pass  for  which  the 
predicted  target  tends  to  be  too  large  because  the  peak  anomaly  is  poorly  sampled  (panels  B  and 
D),  we  find  that  the  recovered  polarizabilities  are  relatively  stable,  although  with  a  greater 
variability  than  cued  data.  The  stability  is  sufficient  to  allow  some  classification  with  dynamic 
data,  using  higher  tolerance  for  library  matching. 
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Figure  12:  Map  of  multiple  dynamic  passes  on  the  IVS. 


Figure  13:  Polarizabilities  of  IVS  items  from  dynamic  data.  (A)  IS02,  (B)  37  mm,  (C)  75  mm  and  (D)  shotput. 


7.2  Classification  performance 

Data  analysis  was  split  between  the  two  survey  areas:  the  forest  where  the  three  man-portable 
systems  acquired  approximately  700  flags  in  cued  mode,  and  the  open  field  dynamic  area,  when 
systems  equipped  with  navigation  and  detection  survey  mode  collected  both  dynamic  and  static 
data. 
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Figure  14:  ROC  curves  for  the  MPV  surveys  at  Spencer  Range. 

(A)  Classification  of  cued  data  from  treed  area,  one  small  ISO  (#2355)  was  missed  as  part  of  a  fragment  pit; 

(B)  In  dynamic  area,  dynamic  data  were  used  to  classify  some  anomalies,  like  a  prescreener,  while  ambiguous 
anomalies  were  analyzed  using  cued  data. 


Classification  results  are  expressed  in  the  fonn  of  a  Receiver  Operating  Curve  (ROC)  in 
Figure  14.  Classification  was  globally  successful  for  the  forested  area  and  met  the  demonstration 
objective:  99%  of  the  TOI  were  found  after  excavating  only  15%  of  the  clutter,  half  of  which 
arising  from  training  excavations.  Moreover,  the  caliber  of  TOI  was  exactly  predicted  92%  of  the 
time  and  differences  in  caliber  were  negligible  98%  of  the  time  (Table  3).  Some  75  mm 
projectiles  were  occasionally  confused  with  medium-sized  ISO.  Target  SR-2640  was  originally 
classified  and  reported  as  a  60  mm  mortar  although  there  were  also  strong  model  candidates  for 
an  IS02  that  would  have  naturally  appeared  8  positions  later  in  the  dig  list.  This  confirms  that 
this  flag  was  not  placed  in  the  dig  list  by  chance.  In  general  prediction  of  the  target  caliber  was 
remarkably  reliable  at  Spencer  Range. 


Table  3:  Targets  with  difference  in  caliber. 


Target  ID 

SR-2031 

SR-2116 

SR-2282 

SR-2547 

SR-2640 

SR-2653 

Validation 

37  mm 

ISO 

medium 

ISO 

medium 

ISO 

medium 

IS02 

(small) 

ISO 

medium 

Prediction 

IS02 

75  mm 

75  mm 

75  mm 

60  mm 

75  mm 

Missed  target  SR-2355 

One  TOI  was  missed  in  the  forested  area.  Target  SR-2355  is  an  IS02  that  was  buried  among 
four  other  metallic  fragments.  An  illustration  of  the  recovered  target  parameters  that  most 
resemble  an  IS02  is  shown  in  Figure  15.  This  result  stem  from  a  2-source  inversion.  It  was 
validated  by  a  geophysicist  after  visual  inspection.  In  retrospect  and  upon  close  scrutiny,  there 
are  signs  that  suggest  potential  issues  with  the  results.  Although  the  data  are  generally  fit,  some 
individual  receivers  are  not  well  accounted  for.  This  may  be  caused  by  the  presence  of  more  than 
2  sources,  or  poor  positioning,  or  some  noise  or  malfunction.  The  recovered  polarizabilities  also 
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show  different  decay  rates  that  could  have  been  interpreted  as  a  tell  tale  for  the  presence  of  a 
combination  of  targets.  This  suggests  that  inversion  with  more  sources  should  have  been 
attempted.  In  retrospect  data  were  re-inverted  with  3  sources  but  the  recovered  polarizabilities 
for  the  IS02  were  still  too  contaminated  by  a  nearby  object  to  allow  clear  classification. 


MPV2  SpcfWoodA_20121 1 14T183031_Bcn_mJJNLrefnaskinv1_Bcn4HR_gdt3 
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Figure  15:  Target  parameters  for  SR-2355,  which  was  missed  during  classification. 

The  upper  left  graph  shows  in  colored  lines  the  recovered  polarizabilities  compared  with  the  reference  IS02 
in  gray.  A  picture  of  the  ground  truth  is  in  the  lower  left.  Data  fits  for  the  different  receiver  components  are 
shown  as  maps  (columns  Obs=observed,  Pred=predicted,  Res=residual  for  each  of  X,  Y  and  Z  components; 
rows  for  each  of  the  cubes)  and  time  decays  on  the  right-side  panels. 


Dynamic  area 

The  dynamic  area  was  processed  under  the  principle  that  dynamic  data  could  be  used  as  a 
pre-screener,  in  which  reliable  data  could  be  interpreted  to  directly  classify  some  anomalies 
without  need  for  a  cued  interrogation,  whereas  ambiguous  anomalies  would  be  further 
characterized  using  cued  data.  In  this  study  we  only  processed  data  for  the  anomalies  that  we  had 
selected  from  our  survey  data.  We  retained  288  anomalies,  including  272  that  were  common  with 
the  EM-61  and  the  2x2  TEMTADS  within  a  margin  of  positional  error.  Some  70  anomalies  were 
not  included  in  our  dig  list  because  their  signal  was  too  low  to  be  considered  of  interest.  As  a 
result  we  omitted  these  anomalies  (these  are  indicated  as  "cannot  analyze"  and  appear  at  the 
beginning  of  the  ROC  curve,  which  was  designed  to  score  a  larger  set  of  anomalies).  Their 
associated  dynamic  data  were  subsequently  inverted  and  analyzed  and  we  confirmed  that  no  TOI 
had  been  missed. 

Dynamic  data  were  extracted  for  the  MPV  288  anomalies  and  inverted  to  attempt 
classification.  Applying  conservative  metrics  for  data  fit  and  spatial  coverage,  half  of  the 
anomalies  were  deemed  to  provide  unreliable  features  for  classification,  while  the  other  half  was 
classified  and  generated  the  first  stage  of  classification.  The  remaining  anomalies  were  classified 
after  inversion  of  their  cued  data.  Classification  was  reliable  and  efficient  with  no-missed  TOI 
and  90%  clutter  rejection. 


ESTCP  MR-201158:  MPV  at  Spencer  Range 


25 


April  2013 


Caliber  was  generally  reliably  predicted  even  with  dynamic  data.  There  were  4  instances 
where  large  differences  in  caliber  occurred  when  based  on  dynamic  data,  though  in  every  case 
the  cued  data  provided  the  correct  caliber:  targets  SR- 1548  and  SR- 1626  were  reported  as  60  mm 
mortars  instead  of  IS02;  target  SR-1729  was  reported  as  75  mm  projectile  instead  of  37  mm, 
although  there  were  valid  model  candidates  for  the  similar-sized  IS02  -  the  associated  hole 
contained  5  pieces  of  metal,  including  one  large  piece,  so  that  the  aggregate  could  be  confused 
with  a  large  target.  Similarly  SR-1502  was  reported  as  a  75  mm  instead  of  a  IS02  with  dynamic 
data  because  there  were  3  pieces  of  metal,  including  one  much  larger  than  the  IS02  (correctly 
predicted  with  cued  data). 

In  conclusion,  the  classification  results  confirm  that  high  quality  data  can  be  collected  with 
the  MPV  in  dynamic  and  static  mode,  and  that  data  processing  methods  are  sufficiently  mature  to 
provide  accurate,  reliable  and  efficient  classification  in  conditions  of  a  site  like  Spencer  Range. 

7.3  Location  and  depth  prediction 

Successful  classification  relies  on  the  capability  to  predict  accurate  target  parameters, 
including  location  and  depth.  As  expected  from  the  previous  section,  we  confirm  that  in  general 
there  was  close  agreement  between  the  predicted  target  depth  and  location  and  the  ground  truth 
validation  (Figure  16).  There  are  some  offsets  for  a  few  TOI;  we  believe  that  these  are  caused  by 
disturbance  in  the  GPS  signal  during  the  ground  truth  excavation;  otherwise  the  MPV  data  could 
not  have  correctly  predicted  such  distant  targets.  Offsets  for  clutter  items  can  also  be  due  to  the 
presence  of  multiple  pieces  of  fragments  around  a  given  flag. 


Figure  16:  Comparison  of  predicted  location  and  depth  with  ground  truth. 

(A)  Observed  and  predicted  depth;  (B)  difference  between  observed  and  predicted  location. 


7.4  Field  survey  statistics  and  production  rate 

The  demonstration  took  place  over  12  days.  A  daily  log  is  detailed  in  Appendix  E.  The  first 
day  was  mostly  spent  on  gathering  the  equipment,  unpacking  and  testing  the  instruments  off  and 
on  the  IVS  and  test  pit.  Measurements  of  training  items  over  the  test  pit  took  a  total  of  3.5  hours. 
The  IVS  was  visited  twice  a  day  for  15  minutes  per  visit  on  average  -  20  minutes  when  dynamic 
data  were  collected. 
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Cued  interrogation  covered  1006  flags,  including  only  24  reacquires  due  to  large  offsets 
between  survey  flag  and  detected  anomaly  or  presence  of  secondary  anomalies.  Daily 
productivity  is  illustrated  in  Figure  17.  On  pure  cued  interrogations  days  the  crew  collected  at 
least  120  flags,  with  two  days  at  150  flags  over  10  hours.  There  were  periods  when  up  to  20 
targets  were  collected  in  one  hour;  when  counting  the  time  for  swapping  of  operator  (every  1-2 
h)  and  batteries  (2-3  h)  the  average  cued  interrogation  rate  was  15  flags  per  hour. 


Figure  17:  Graphical  summary  of  daily  data  collection  production  rate. 

Number  of  cued  flags  is  shown  as  a  function  of  time  spent  in  the  field  each  day:  Cued  interrogation  (red), 
detection  (blue)  and  interruptions  (red  plateaus:  rain  stopped  operations  on  June  13,  14,  15  &  21  for  several 
hours;  2  h  visitors  demonstration  on  June  19). 

The  production  rate  was  on  par  with  expectations.  The  minimum  requirement  was  to  acquire  at 
least  5  soundings  per  flag  (applicable  for  80%  of  the  flags)  and  to  regularly  characterize  the 
background  noise,  taking  one  measurement  on  the  ground  and  in  the  air  every  seventh  flag  and 
before  and  after  swapping  batteries.  On  average,  an  anomaly  was  characterized  with  6  soundings 
in  the  forest  and  8  in  the  dynamic  area,  where  the  crew  was  instructed  to  track  neighboring 
anomalies  to  account  for  differences  in  the  picked  locations  between  the  MPV  survey  and  the 
other  sensors.  One  sounding  typically  took  15  seconds  (7  seconds  for  acquisition,  the  rest  for 
interpreting  the  data  and  moving  the  sensor  head).  In  total  7200  static  soundings  were  acquired 
for  cued  interrogation,  350  files  for  backgrounds,  and  570  files  for  the  IVS  and  test  pit. 

Dynamic  collection  was  started  on  the  fourth  day,  after  another  survey  crew  had  cleared 
away  from  the  area.  The  1.3 -acre  survey  area  was  covered  in  approximately  20  hours  using  two 
field  operators  most  of  the  time.  A  third  person  occasionally  helped  laying  survey  lines  to  relieve 
the  crew  and  minimize  delay.  Lines  were  interrupted  after  30  m  in  order  to  save  the  file  (total  of 
270  files),  take  a  short  rest  and  monitor  the  survey  path  on  the  display  to  control  coverage.  Some 
gaps  between  lines  or  near  the  edge  of  the  survey  were  observed  and  therefore  some  partial  lines 
were  reacquired  at  the  end  of  the  survey  (two  hours),  using  the  control  display  to  locate  the 
sensor  head  relative  to  previous  survey  paths. 


ESTCP  MR-201158:  MPV  at  Spencer  Range 


27 


April  2013 


8.0  COST  ASSESSMENT 

Time  and  resources  were  tracked  for  each  task  to  assess  the  cost  of  deploying  the  technology 
at  future  live  sites.  The  technology  was  still  being  improved  at  the  time  the  demonstration  took 
place.  In  particular,  use  of  a  new,  dedicated  AHRS  on  the  beacon  boom  required  modifications  to 
the  DAQ  software  (made  by  G&G  Sciences)  and  to  the  beacon  data  processing  algorithms,  and 
led  to  additional  studies.  This  demonstration  was  also  the  first  time  that  detection  data  were 
collected  and  used  for  target  picking,  which  resulted  in  on-site  development  work  for  the 
assimilation,  characterization  and  interpretation  of  dynamic  data.  Sensor  maintenance  and  data 
processing  costs  are  therefore  higher  than  expected  for  production-type  surveys. 

8.1  COST  MODEL 

A  cost  model  for  the  Spencer  Demonstration  is  proposed  in  Table  4.  It  is  based  on  a  burdened 
hourly  rate  of  $100  for  any  of  the  personnel  involved.  The  field  study  was  conducted  with  three 
people:  one  field  technician,  one  field  geophysicist  with  ample  experience  with  the  MPV 
technology,  and  one  research  geophysicist  who  managed  the  study,  occasionally  helped  with  data 
collection  and  handled  all  data  processing  tasks.  Twelve  days  were  spent  on  site. 


Table  4:  Cost  model  for  MPV  demonstration  at  Spencer  Range. 


Cost  Element 

Data  to  be  Tracked 

Estimated 

Unit 

Time 

Estimated 

Total 

Hours 

Estimated 
Total  Cost 

Survey  preparation 

Sensor 

Unit:  $  Cost 

maintenance 

•  MPV  maintenance  and  modifications 

$10,000 

Pre -survey 

Personnel:  Geophysicist 

■n 

■I 

activities 

•  Demonstration  plan  and  coordination 

•  Preparation  of  survey  data 

dEHM 

Development 

Personnel  required:  Geophysicist 

time 

Time  to  prepare  assimilation  of  detection  data 

10  h 

$1,000 

Time  to  test  target  picking  algorithms 

lOh 

$1,000 

Time  to  integrate  &  test  2nd  AHRS  for  beacon 

40  h 

$4,000 

Field  survey 

Mobilization 

Cost  to  mobilize  to  site:  3  people 

and 

•  Flight,  car,  hotel,  per  diem 

lOh 

60  h 

$16,000 

demobilization 

•  Shipping 

2  h 

4  h 

$3,000 

•  Daily  travel 

2  h 

72  h 

Rentals, 

Survey  equipment  rental  (GPS) 

$3,000 

materials  and 

Material  supplies 

3  h 

$1,000 

miscellaneous 

Miscellaneous  tasks  and  interruptions 

20  h 

$2,000 

Instrument 

Field  crew:  Geophysicist  +  technician 

setup 

•  First  day:  assemble,  set  up  and  test  pit 

8  h 

16  h 

HIS 

•  Typical  day  (GPS  set  up  and  IVS  surveys) 

1  h 

24  h 

mmm 

•  Analyze  IVS  data  (Geophysicist) 

0.5  h 

6  h 

$600 
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Data 

collection  for 
detection 

Personnel  :  Field  crew  of  2 

Personnel :  Geophysicist 

Crew:  Collect  &  record  data  per  acre 
(total  1.3  acres) 

4  h 

15  h 

n 

$4,600 

Data 

collection  for 
cued  survey 

Personnel  :  Field  crew  of  2 

Personnel :  Geophysicist 

Crew:  Acquire,  collect  and  record  data  per  flag 
(total  1,000  flags) 

3.6  min 

15  h 

120  h 

$13,500 

Pre-processing 
and  QC 

Personnel  required:  Geophysicist 

Detection:  Cost  per  acre 

Cued  data:  Cost  per  flag 

Cued  data:  Additional  processing,  post  survey 
(study  drift  of  beacon  boom  AFIRS) 

8  h 

2.5  min 

lOh 

40  h 

40  h 

Data 

extraction 

Personnel  required:  Geophysicist 

Time  to  built  detection  map  per  acre 

lOh 

13  h 

$1,300 

Anomaly 

selection 

Personnel  required:  Geophysicist 

Time  to  establish  anomaly  selection  threshold 
and  pick  anomalies  in  dynamic  data 

m 

11  h 

$1,100 

Data  processing 

l  of  detection  data 

Data 

extraction 

Personnel  required:  Geophysicist 

Time  to  extract  and  mask  dynamic  data 

2  min 

lOh 

$1,00 

Parameter 

extraction 

Personnel  required:  Geophysicist 

Time  for  inversion  &  QC 

3  min 

15  h 

$1,500 

Classifier 

training 

Personnel  required:  Geophysicist 

Time  to  build  feature  library  for  dynamic  data 

2  min 

lOh 

$1,000 

Classification 
and  dig  list 
generation 

Personnel  required:  Geophysicist 

Time  required 

2  min 

lOh 

$1,000 

Data  processing  of  cued  interrogation  data 

Data 

extraction 

Personnel:  Geophysicist 

Time  to  extract  and  mask  cued  data 

1  min 

17  h 

$1,700 

Parameter 

extraction 

Personnel:  Geophysicist  in  training  +  expert 
Time  for  inversion  &  QC 

3  min 

65  h 

$6,500 

Classifier 

training 

Personnel:  Geophysicist 

Time  to  identify  features  and  potential  TOl 

1.5  min 

25  h 

$2,500 

Classification 
and  dig  list 
production 

Personnel:  Geophysicist 

Time  to  prepare  memo,  apply  classifier  and 
assimilate  ground  truth 

1.5  min 

30  h 

$3,000 
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8.2  COST  DRIVERS 

The  MPV  was  developed  to  provide  a  moderate  cost,  reliable,  portable  sensor  with  advanced 
discrimination  capabilities  that  can  operate  at  sites  with  challenging  surveying  conditions.  As  a 
portable  system,  deployment  logistics  and  costs  for  transport  and  operation  are  relatively  lower 
than  those  of  towed  arrays  or  other  vehicular-based  systems.  The  primary  costs  are  incurred  for 
labor  and  travel  for  the  operators,  and  the  primary  cost  driver  becomes  the  duration  of 
deployment,  directly  related  to  the  acreage  to  be  surveyed  as  well  as  the  difficulty  of  the  terrain 
(steep,  rocky,  very  uneven,  and  wooded  terrain  can  take  somewhat  longer  to  survey  because  it  is 
more  difficult  to  hike  across  these  areas). 

8.3  COST  BENEFIT 

The  primary  driver  for  developing  the  MPV  is  to  make  discrimination  feasible  at  a  wide 
range  of  sites  where  field  conditions  prohibit  the  use  of  cart-based  systems,  and  for  small-scale 
deployment  where  a  small  area  needs  to  be  surveyed  or  where  anomalies  need  to  be  resurveyed 
at  a  lower  cost  than  a  cart-based  system. 
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9.0  IMPLEMENTATION  ISSUES 

The  MPV  is  based  on  technology  that  is  not  expected  to  be  affected  by  any  particular 
regulation.  The  MPV  has  low  impact  on  its  environment  owing  to  its  handheld  deployment 
mode,  which  leaves  surface  impact  and  requires  minimal  site  preparation  and  vegetation 
clearance.  Some  DoD  sites  do  regulate  radio  frequency,  which  could  affect  use  of  GPS. 
However,  there  is  no  reliance  on  GPS  for  cued  interrogation  through  the  beacon,  while  detection 
surveys  can  afford  lower  spatial  accuracy  requirements  with  cotton  thread  and  spool. 

The  original  implementation  objectives  were  to  prove  that  high  quality  data  could  be 
collected  and  support  reliable  classification,  to  verify  that  such  perfonnance  could  be  achieved 
within  a  handheld  form  factor  able  to  sustain  field  use,  and  to  assess  viable  deployment  modes. 
The  technology  was  tested  for  detection  and  cued  interrogation.  The  quality  of  the  collected  data 
was  sufficient  to  detect  all  TOI  and  correctly  classify  99%  of  the  TOI,  thus  exceeding  the 
demonstration  objectives.  The  production  rate  for  cued  interrogation  improved  by  more  than 
30%  without  compromising  classification  performance.  Maneuverability  and  ruggedness  were 
considered  to  be  adequate  by  our  team,  though  our  newer  field  member  had  a  smaller  stature  and 
found  the  detection  survey  sweep  to  be  uncomfortable  -  the  handle  could  be  modified  in  the 
future.  Our  crew  was  able  to  survey  for  up  to  10  hours  a  day  by  regularly  swapping  roles 
between  operating  the  MPV  and  carrying  the  DAQ  backpack.  Commercial  users  have  not  yet 
tested  the  technology. 

The  MPV  tested  in  this  study  is  a  second-generation  prototype  built  by  G&G  Sciences.  It  is 
not  commercially  available  and  there  currently  exist  only  one  government-owned  unit,  which 
maintenance  has  to  be  sourced  to  G&G.  Basic  components  are  commercially  available  -  receiver 
cubes  and  National  Instrument  DAQ  computer  and  receiver  modules  -  while  receiver  filters, 
transmitter  coils  and  switches  and  DAQ  software  (EM3DAcquire)  are  G&G  custom  builds.  The 
MPV  requires  an  attitude  and  heading  sensor;  there  is  no  standard  communication  protocol  and 
few  models  are  currently  supported,  which  can  incur  significant  costs  if  alternative  attitude 
sensors  are  to  be  integrated  such  that  they  are  compatible  with  EM3DAcquire.  Any  GPS  can  be 
used  as  long  as  it  is  set  to  send  a  standardized  data  sequence  (e.g.,  NEMA  formats). 

The  MPV  is  relatively  straightforward  to  operate  in  the  field  for  general  use.  One  crew 
member  was  new  to  UXO  surveys  and  was  quickly  trained  to  collect  data  and  interpret  the 
recorded  signals,  in  particular  for  locating  the  signal  origin  and  detecting  malfunctioning 
receivers.  Advanced  manipulation  of  the  user  interface  for  setting  up  and  troubleshooting 
communication  with  peripherals,  GPS  and  attitude  sensor  requires  more  familiarity  with  the 
system.  Experience  with  setting  up  a  GPS  is  useful  for  the  detection  survey. 

Instructions  on  survey  protocols,  such  as  acquisition  parameters  and  cued  survey  patterns,  are 
contained  in  the  YPG  and  Camp  Beale  reports.  General  guidance  on  EM3DAcquire  can  be  found 
on  the  Geometries  website  for  the  MetalMapper  sensor,  which  uses  similar  underlying 
technology.  At  this  point  we  would  recommend  that  an  experienced  geophysicist  be  associated 
with  any  data  collection  to  guide  field  operators  and  frequently  verify  data  quality. 
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APPENDICES 


Appendix  A:  Health  and  Safety  Plan  (HASP) 

Health  and  safety  procedures  were  followed  as  indicated  below,  and  also  complied  with  the 
ESTCP  guidance  for  this  demonstration. 

-  Applicable  local,  state,  and  federal  health  and  safety  laws  and  regulations 

Field  staff  complied  with  health  and  safety  requirements  in  accordance  with  Code  of  Federal 
Regulations  (CFR),  Part  29,  Section  1910.120  and  any  site-specific  requirements  as  noted  during 
site  orientation  or  other  direction  provided  by  ESTCP  and  Department  of  fish  and  Game 
representatives.  Although  29  CFR  1910.120  pertains  to  personnel  conducting  activities  at  known 
or  suspected  hazardous  waste  sites  and  may  not  be  directly  applicable  to  the  planned  activities, 
the  code  provides  a  reasonable  framework  safe  work  practices. 

Potential  for  worker  exposure  to  hazardous  materials  and/or  other  hazards:  None  or 
minimal.  Any  site-specific  hazards  also  were  to  be  briefed  during  orientation. 

Physical  requirements  are  expected  of  workers:  Basic  fitness,  heat  resilience. 

Number  of  people  required  to  operate  the  technology:  Two  field  operators  and  one  QC 
person  (PI) 

Technology’s  history  of  breakdowns  or  accidents:  No  issues  to  date. 

Potential  effects  from  the  transporting  of  equipment,  samples,  wastes,  or  other 
materials  associated  with  the  technology:  All  components  of  the  technology  are  inert,  to 
the  exception  of  Fi-ion  batteries  that  must  be  shipped  by  ground  according  to  federal 
regulation. 

Impact  of  technology  on  surrounding  environment:  None.  Technology  is  non  destructive 
and  man  portable. 

Closest  medical  facility:  There  are  two  facilities  near  McMinnville,  TN: 

River  Park  Hospital:  Focated  at  1559  Sparta  Road,  McMinnville,  Tennessee,  37110. 
Telephone:  (931)  815-4000 

Middle  Tennessee  Surgical  Care:  Focated  at  145  Health  Way,  McMinnville,  Tennessee, 
37110.  Telephone:  (800)  364-5210  or  (931)  507-6872 

The  facilities  are  indicated  on  the  map  of  Figure  18  and  Figure  19  relative  to  McMinnville 
and  to  the  test  area. 
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Figure  18:  Location  of  medical  facilities:  river  Park  in  NE  and  Surgical  Care  in  E. 
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Figure  19:  Regional  map  with  Spencer  Range  location  relative  to  McMinnville. 
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Appendix  B:  Points  of  Contact 

Points  of  contact  (POCs)  involved  in  the  demonstration  and  their  contact  information  are 
presented  in  Table  5. 


Table  5:  Points  of  Contact  for  the  MPV  Demonstration. 


POINT  OF 
CONTACT 
Name 

ORGANIZATION 

Name 

Address 

Phone 

Fax 

E-mail 

Role  in  Project 

Dr.  Nicolas 
Lhomme 

Black  Tusk 
Geophysics,  Inc. 

112  A,  2386  East 
Mall,  Vancouver,  BC 
V6T  1Z3,  Canada 

Tel:  604-428-3382 

Nicolas.  lhomme(a>bt2eonhvsics.  com 

PI 

David  George 

G&G  Sciences,  Inc. 

873  23  Rd 

Grand  Junction,  CO 
81505 

Tel:  (970)  263-9714 

Fax: (970)  263-9714 
dgeorge(S}ggsciences  .com 

Sensor 

manufacturer 

Dr.  Herb  Nelson 

ESTCP  Program 
Office 

ESTCP  Office 

901  North  Stuart 
Street,  Suite  303 
Arlington,  VA 
22203-1821 

Tel:  703-696-8726 
Herbert.Nelson(2>osd.mil 

ESTCP  Munitions 
Management 
Program  Manager 
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Appendix  C:  Detection  range  with  MPV  survey 


We  determine  the  spatial  coverage  requirement  for  an  MPV  survey  by  assessing  the  MPV 
footprint  with  a  particular  focus  on  the  signal  on  its  side  receivers.  We  simulate  37  mm  projectile 
at  various  horizontal  offsets  and  depths  in  horizontal  position  (weakest  signal)  and  vertical 
orientation  (shortest  spatial  decay);  the  we  compare  with  strong  background  signal  (Beale  soil). 
Polarizabilities  are  extracted  from  static  data  (proven  to  have  similar  decay  as  dynamic  one  in  0- 
2.7  ms  excitation  range)  and  background  noise  is  taken  from  measurements  Camp  Beale,  where 
their  intensity  was  above  normal. 
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Figure  20:  SNR  cross  section  for  different  target  offset  and  depth  relative  to  side  receiver. 

The  target  is  a  37  mm  projectile  place  in  horizontal  orientation.  Noise  is  based  on  Camp  Beale  magnetic  soil. 
Reference  receiver  is  MPV  side  cube  #2.  Negative  coordinates  correspond  to  inner  MPV.  SNR=1  when  signal 
and  background  have  equal  amplitude;  target  are  detectable  when  SNR>1.6  (signal  is  60%  larger  than  noise). 


The  relative  amplitude  of  target  signal  and  background  noise  (Signal  to  Noise  Ration  or 
SNR)  is  illustrated  in  Figure  20  for  a  typical  37  mm  in  horizontal  orientation.  The  figure  shows 
the  SNR  for  different  positions  and  depth  of  the  target  relative  to  a  side  receiver  on  the  MPV. 
When  the  target  is  below  the  receiver  (Distance=0),  it  is  detectable  at  35  cm  depth  as  signal 
exceeds  noise  by  60%  (SNR=1.6  contour).  Similar  detection  signal  would  be  observed  with  10 
cm  lateral  offset.  At  20  cm  offset  detection  would  remain  clear  at  30  cm  depth.  If  the  target  was 
vertical  it  would  be  clearly  detected  at  35  cm  depth  with  a  lateral  offset  of  up  to  15  cm  (Figure 
21).  Therefore  we  can  assume  that  a  10  cm  target  offset  would  not  hann  detection  of  a  37  mm 
target. 


Simulations  for  a  105mm  projectile  suggest  detection  in  Camp-Beale-type  conditions  at  70 
cm  with  10  cm  offset  (Figure  22).  Detection  depth  would  increase  by  20-30  cm  at  a  site  with  low 
background  response  (Figure  23).  Overall  detection  depth  is  generally  unaffected  when  the  target 
is  up  to  10  cm  away  from  the  sensor  head  and  therefore  the  sensor  footprint  is  approximately  70 
cm  (50  cm  diameter  sensor  head). 


ESTCP  MR-201158:  MPV  at  Spencer  Range 


36 


April  2013 


-0.1  0  0.1  0.2  0.3  0.4  0.5 

Distance  from  receivsr  cube  (m) 


Figure  21:  37  mm  target  in  vertical  orientation. 


Figure  22:  SNR  cross  section  for  105  mm  projectile;  (A)  Horizontal;  (B)  Vertical. 
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Appendix  D:  Root  Cause  Analysis  for  Missed  Detection 

This  document  discusses  the  causes  for  missing  three  targets  in  the  MPV  detection  list  that 
was  submitted  during  the  Spencer  Range  survey.  The  following  analysis  shows  that  the  targets 
were  detectable  and  that  the  error  came  from  the  rushed  interpretation  stage. 

1.  Background 


The  raw  GPS  data  were  valid.  The 
MPV  and  GPS  data  were  re-imported  to 
produce  a  new  detection  map,  using  our 
own  conversion  from  longitude-latitude 
to  UTM  coordinates  as  part  of  our  data 
import  process.  Because  of  the  tight 
schedule,  we  tried  to  save  time  by 
loading  only  the  data  that  lay  within  the 
survey  boundaries  -  doing  so  would 
reduce  the  volume  of  data  and 
bookkeeping  of  targets  outside  the 
survey  boundaries. 
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Figure  24:  Detection  map  of  Spencer  dynamic  area;  MPV 
total  field  data  at  1.4  ms.  The  survey  boundary  is  indicated 
with  the  green  line. 


The  following  analysis  shows  that  two  of  the  missed  targets  were  located  within  0.15  m  of 
the  survey  boundary;  as  a  result,  their  corresponding  anomalies  were  truncated  and  did  not  meet 
the  detection  criteria.  The  third  anomaly  was  merged  with  its  nearby  anomaly  and  should  have 
been  picked  with  adequate  quality  control. 


This  demonstration  was  the  first  time  that  dynamic  data  was  collected  and  used  for  target 
picking.  Data  analysis,  mapping  and  target  picking  were  performed  during  the  Spencer  Range 
deployment.  Time  was  critical  as  the  target  picks  had  to  be  submitted  to  a  third  party  for  review, 
and  then  picks  had  to  be  surveyed  in 
cued  mode  by  the  MPV  field  crew. 

Handling  of  dynamic  data  was  initially 
slow,  as  each  survey  line  was  first 
examined  to  verify  data  quality  and 
coverage,  and  then  added  to  the  other 
lines  to  build  up  a  map.  Unfortunately 
that  initial  detection  map  was  built  with 
incorrect  spatial  coordinates  due  to  an 
error  with  the  internal  MPV  software  in 
converting  the  GPS  longitude-latitude  to 
UTM  coordinates.  This  caused 
irresolvable  spatial  offsets  that  made  the 
first  submitted  target  list  unusable  and 
caused  significant  delay  in  validating 
the  MPV  detection  picks  relative  to 
other  sensor  picks. 
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2.  Target  detection  process 

Dynamic  survey  data  were  assimilated,  filtered 
and  interpreted  to  identify  anomalies  that  required 
further  investigation.  Anomaly  selection  was  based 
on  quantitative  metrics  derived  from  simulations 
and  analysis  of  the  worst  case  scenario  for  the 
expected  targets.  At  Spencer  Range  the  detection 
objective  was  to  find  37  mm  projectiles  at  0.35  m 
depth.  The  typical  response  of  that  target  type  can 
be  simulated  to  predict  the  anomaly  amplitude  and 
footprint.  Model  polarizabilities  were  extracted  from 
static  data;  static  polarizabilities  have  similar 
amplitude  and  decay  rate  as  dynamic  ones  with  2.7 
ms  excitation,  as  shown  in  Figure  25  with  static  and 
dynamic  data  from  the  Spencer  IVS. 


Figure  25:  Recovered  polarizabilities  for  a 
37  mm  projectile  based  on  inversion  of 
static  (grey  lines)  and  dynamic  (colors) 


The  target  response  is  shown  in  horizontal  position  (weakest  signal)  and  vertical  orientation 
(shortest  spatial  decay)  in  Figure  26. 


Figure  26:  Predicted  response  for  37  mm  projectile:  horizontal  (A);  vertical  (B). 


The  ability  to  detect  a  target  depends  on  its  signal 
amplitude  relative  to  the  background  noise  variability. 
Noise  in  a  detection  survey  originates  from  intrinsic 
sensor-electronics  noise,  environmental  noise  and 
sensor  motion  relative  to  the  ground  (if  there  is  a  soil 
response  due  to  viscous  remanent  magnetization).  The 
variability  at  the  Spencer  Range  IVS  is  illustrated  in 
Figure  27,  where  the  total  field  data  at  1.4  ms  is  shown 
over  a  target-free  area  (unleveled  data).  The  background 
signal  variability  (1  □)  for  the  total  field  amplitude  at  1 .4 
ms  was  0.09  mV/A.  After  leveling,  the  data  retains  that 
variability  and  one  could  assume  that  a  target  would  be 


Figure  27:  Distribution  of  background 
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detectable  when  its  anomaly  consistently  exceeded  2-3  standard  deviations  over  a  given  surface 
area.  Applying  that  automated  method  at  Spencer,  we  obtained  a  list  that  we  ranked  according  to 
the  maximum  amplitude  of  the  1.4  ms  total  field  channel.  Then  we  computed  the  ratio  of  the 
maximum  amplitude  for  the  0.5  and  1.4  ms  channels  and  eliminated  the  fastest  decaying 
anomalies  for  the  targets  with  the  smallest  amplitudes. 


3.  Missed  targets 

For  the  following  analysis  the  data  were  re¬ 
imported  with  wider  bounds.  The  target  picking 
algorithm  used  the  same  thresholds,  which  were 
based  on  a  minimum  spatial  extent  at  given  signal 
amplitude.  The  1.4  ms  time  channel  was  used 
under  the  assumption  that  small  metallic  scrap  has 
fast  temporal  decay. 

The  first  missed  target  was  located  at  UTM 
coordinates  (Easting  =  635341.98  m,  Northing  = 
3939191.38  m).  The  corresponding  signal  anomaly 
is  shown  in  Figure  28  with  label  number  360 
(ESTCP  label  SR- 1766).  The  anomaly  sits  at  the 
survey  boundary.  The  target  was  missed  by  our 
analysis  because  the  anomaly  was  truncated  when 
importing  only  the  data  within  the  survey  limits. 
As  a  result  the  anomaly  extent  was  smaller  than 
the  minimal  extent  threshold. 


X-  635400 


Figure  28:  Detection  map  near  the  1st  missed 
target  (360).  The  survey  boundary  is  indicated 
with  a  green  line. 


The  second  anomaly  was  located  at  UTM  (635380.12  m,  3939075.22  m).  Its  corresponding 
anomaly  is  shown  in  Figure  29A,  label  271  (SR- 1506).  The  anomaly  was  missed  for  the  same 
reason  as  the  previous  one.  The  target  pick  lies  outside  of  the  survey  boundary. 


Figure  29:  Detection  map  for  the  2nd  missed  anomaly  (A,  marker  #271  on  the  green  survey  boundary)  and  3rd 
one  (B,  secondary  anomaly  1  m  to  the  North  of  marker  #6). 
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The  third  target  was  located  at  UTM(635373.05  m,  3939101.52  m).  Its  corresponding 
anomaly  is  shown  in  Figure  29B,  where  one  large  or  two  adjoined  anomalies  relate  to  label  6. 
The  missed  target  location  (SR- 1555)  is  situated  1  m  to  the  North  of  label  6  (SR- 1554).  The 
anomaly  was  missed  because  the  two  anomalies  were  grouped  together  because  of  a  low 
amplitude  threshold.  Visual  quality  control  would  have  picked  the  second  target. 

4.  Improving  quality  control 

All  three  targets  were  missed  because  the  process  was  rushed.  The  first  two  anomalies  would 
have  been  picked  if  wider  survey  bounds  would  have  been  applied.  For  the  third  missed  target, 
the  original  mistake  was  to  use  a  small  color  range  to  enhance  small-amplitude  anomalies.  Closer 
scrutiny  shows  that  there  are  two  targets  and  that  they  appear  to  be  clearly  separated  when 
displaying  the  Z-component  data  (or  the  total  field)  with  a  wider  color  range  and  examining  the 
transverse  component  data  (Figure  30). 


Figure  30:  Re-examination  of  the  3rd  missed  target;  Data  for  Z  (A)  and  Y  (B)  component  receivers. 


5.  Conclusion 

Moving  forward,  better  detection  of  potential  targets  can  be  achieved  by  investing  more 
effort  in  examining  detection  maps.  Several  maps  could  be  utilized  to  include  the  information 
from  different  time  channels  and  different  components  of  the  data.  Comparison  of  multiple  time 
channels  can  eliminate  fast-decaying  anomalies  and  separate  large-amplitude  anomalies  like  SR- 
1554/1555.  Transverse  component  data  help  locate  a  target  given  that  the  X-component  receiver 
data  generally  change  from  positive  to  negative  amplitude  near  the  target  along  the  East- West 
direction,  while  Y  data  show  the  plus-minus  switch  in  the  North-South  direction.  Ideally  this  rich 
information  content  should  be  objectively  assessed  with  quantitative  metrics  rooted  in  physics. 
Each  anomaly  above  a  given  low-threshold  value  could  be  inverted  with  a  point-dipole-type 
model  to  substantiate  that  analysis.  This  proposed  method  requires  high  spatial  coverage  and 
accurate  positioning  to  limit  distortion  of  anomaly  extent.  These  conditions  cannot  always  be 
met.  Alternatively,  every  anomaly  above  a  given  amplitude  threshold,  irrespective  of  spatial 
extent,  can  be  selected  as  a  potential  target.  In  this  case  there  is  no  prescreening  of  near-surface 
metallic  clutter  and  classification  is  only  applied  to  cued  interrogation  data. 
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Appendix  E:  Field  notes 


Monday,  June  1 1 :  Travel  (N.  Lhomrne  +  J.  Jacobson)  and  prepare  target  list  for  cued 
interrogation  (conversion  and  fonnatting).  Fly,  get  rental  car  and  pick  up  all  survey  equipment 
(everything  fits  in  one  Suburban  car) 

Tuesday,  June  12:  Bring  all  survey  equipment  to  site  (after  delay  with  shipment).  Assemble 
equipment  and  verify  operation  (2  hours),  setup  survey  points  for  GPS  navigation.  Cued 
interrogation  survey  at  IVS  (20  min)  and  at  training  pit  (2  hours). 

Wednesday,  June  13:  IVS  (20  min),  training  pit  (lh)  and  start  of  survey  in  woods  (area  A,  East 
of  access  road).  Several  rain  interruptions.  50  targets 

Thursday,  June  14:  J.  Gayer  joins  survey  team  and  gets  trained  at  operating  the  MPV:  theory, 
motivation  behind  SOP,  interpretation  of  data  on  control  screen  to  adjust  survey  pattern, 
programming  of  survey  parameters,  control  of  proper  operation.  Start  survey  at  IVS  and  in 
woods  with  3,  then  2  persons.  Lhomine  starts  daily  QC  cued  interrogations:  import  data,  verify 
beacon  positioning  accuracy,  import  in  UXOLab  and  invert  data,  verify  adequate  spatial 
coverage  of  anomalies  for  stable  target  parameter  recovery.  Several  rain  interruptions.  70  targets 

Friday,  June  15:  Cued  interrogation  by  Jacobson-Gayer.  Lhornme  prepares  detection  survey 
(program  lines)  and  QC  cued  interrogations.  Thunderstorms  and  rain. 

Sunday,  June  17:  Finish  wood  area  A  (2h)  and  start  dynamic  survey  with  3  people.  All  train  at 
dynamic  collection  on  IVS  (for  every  day  of  dynamic  detection).  Lhornme  lays  out  1.5-m-spaced 
lines  and  visually  checks  that  survey  speed  and  spatial  coverage  is  adequate.  30%  of  area  is 
covered  (5-6h).  Lhornme  starts  importing  dynamic  data  and  mapping  to  verify  adequate  spatial 
coverage. 

Monday,  June  18:  Dynamic  survey  of  middle  area  with  2  field  operators  and  3  person  lying  lines. 
Reduce  spacing  to  approximately  1-1.2  m  to  make  walking  and  sweeping  sensor  easier.  Lhornme 
imports  and  maps  dynamic  data 

Tuesday,  June  19:  Dynamic  collection  of  southerner  portion.  Recollection  of  lines  with  wide  line 
spacing  and  short  lines.  Demonstration  of  detection,  cued  interrogation  and  data  processing  to 
USAGE  visitors.  End  of  dynamic  detection  and  move  to  cued  survey  in  woods.  Lhornme 
prepares  detection  maps. 

Wednesday,  June  20:  Cued  interrogation  in  woods  (140  targets).  Lhornme  prepares  detection 
maps  and  submits  target  detection  list  based  on  simulated  response  of  37-min  projectile  buried  at 
34-cm  depth. 

Thursday,  June  21:  Cued  interrogation  in  woods  (150  targets).  Lhornme  re-imports  all  detection 
data  due  to  problematic  geographical  coordinate  conversion  in  EM3D,  re-applies  detection 
algorithm  and  submits  corrected  detection  list. 
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Friday,  June  22:  Cued  interrogation  ends  in  woods  (88  targets)  and  starts  in  open  field.  Lhomme 
prepares  cued  interrogation  list  based  on  main  list  supplied  by  Amy  Walker  (272  targets  detected 
by  EM-61,  TEMTADS  2x2  and  MPV)  and  16  additional  targets  of  interest  for  the  MPV. 

Saturday,  June  23:  Cued  interrogation  in  open  field  (153  targets).  Lhomme  QC  cued  data 

Sunday,  June  24:  End  of  main  cued  interrogations  (82  targets).  Lhomme  QC  data  and  requests  16 
recollects  in  woods  (targets  detected  near  flag,  over  0.5  m  away)  and  8  in  open  field  (extend 
pattern  for  ambiguity  on  peaks  of  interest).  Demobilize,  pack  and  ship  equipment. 

Monday,  June  25:  Travel  back  home  and  report. 
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Appendix  F:  Classification  Decision  Memorandum 


Classification  is  performed  using  the  standard  UXOLab  software.  The  cued  data  were 
inverted  using  standard  inversion  parameters  as  proposed  in  the  demonstration  plan.  Results 
were  reviewed  to  verify  that  the  recovered  models  fit  the  observed  data.  When  necessary, 
inversions  were  redone  after  altering  model-search  bounds  or  masking  out  some  data  in  order  to 
improve  the  fits.  Models  that  did  not  fit  the  data  of  interest  were  rejected  while  all  other  plausible 
models  were  kept  for  classification.  A  spatial  representation  of  the  recovered  models  is  shown  in 
Figure  3 1,  for  which  size  and  decay  parameters  were  derived  from  inverted  polarizabilities. 


Figure  31:  Distribution  of  target  size  and  decay  parameters  relative  to  reference  UXOs. 
Symbol  colors  reflect  the  likelyhood  that  anomalies  are  potential  UXO  (based  on  library  misfit). 


1.  Unclassifiable  anomalies 

Every  field  anomaly  was  adequately  fitted  with  a  model  that  either  described  a  metallic 
object  or  a  background  response  from  the  soil  (which  has  moderate  magnetic  viscous 
remanence).  However,  some  anomalies  appeared  to  be  on  the  edge  of  the  cued  survey  and 
yielded  target  parameters  that  loosely  resembled  UXO.  These  anomalies  were  failed  because 
recovered  parameters  could  not  be  fully  trusted  to  dismiss  these  targets  as  potential  UXO. 
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Figure  32:  Failed  anomaly  on  edge  of  survey. 
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Figure  33:  Failed  anomaly  on  edge  of  survey. 
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Figure  34:  Failed  anomaly  on  edge  of  survey. 


2.  Classification  approach  and  features 

Classification  is  based  the  recognition  of  typical  UXO  features.  Potential  UXO  are  identified 
because  their  recovered  target  parameters  -  or  signature  -  resemble  known  UXO,  or  there  are 
multiple  occurrences  of  metallic  objects  with  highly  similar  features.  The  classification  features 
are  the  three  time-varying,  orthogonal  polarizabilities  that  are  derived  from  the  geophysical 
inversion  of  field  data. 


3.  Training  data  selection 

A  first  set  of  anomalies  was  selected  for  training  through  cluster  analysis  of  similar  model 
signatures.  Potential  targets  of  interest  included  several  circular,  elongated  metallic  objects  of 
similar  or  smaller  size  than  the  ISO  that  was  used  at  Camp  Beale,  as  well  as  fuze-like  objects  and 
larger  targets  up  to  a  155  mm  projectile  (Figure  35).  The  second  pass  was  based  on  training  items 
that  were  selected  by  the  geophysicist  who  perfonned  the  inversion  QC.  These  items  were 
selected  because  they  resembled  known  targets  such  as  full  and  partial  60  mm  mortars.  After  this 
training  we  resolved  an  earlier  confusion  between  the  “IS02”  and  the  medium  ISO,  which  both 
seem  to  differ  from  the  ISO  encountered  at  Beale.  The  third  round  of  training  was  requested  to 
clarify  that  there  was  no  Beale  ISO,  that  there  were  indeed  60  mm  mortars,  and  to  confirm  that 
multi -target  such  as  37  mm  plus  frag  could  be  predicted.  The  training  data  summary  table  is 
shown  below. 
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Table  6:  Training  data  summary. 


Target 

ID 

Request 

Similar  to 

Validated 

Reason  for  selection 

SR-2003 

1 

fuze 

frag 

SR-2242 

1 

fuze 

frag 

SR-2449 

1 

fuze 

frag 

SR-2082 

1 

very  small  ISO 

frag,  frag 

SR-2108 

1 

small  ISO 

frag 

SR-2124 

1 

small  ISO 

frag 

SR-2208 

1 

small  ISO 

frag 

SR-2079 

1 

ISO  (smaller  L23) 

frag 

SR-2231 

1 

ISO  slower  decay 

frag,  frag 

predicted  2  tg 

SR-2232 

1 

ISO  very  slow  decay 

frag,  frag,  frag 

loglin  decay 

SR-2076 

1 

IS02 

frag 

IS02  shared  with  #2069? 

SR-2099 

1 

IS02? 

frag,  frag,  frag 

not  perfect  fit  =  sign  of  3  frags 

SR-2598 

1 

IS02 

frag 

SR-2616 

1 

IS02 

frag 

SR-2492 

1 

37  or  IS02 

frag 

different  models  GPS/beacon 

SR-2262 

1 

75mm 

75mm+frag 

slightly  faster  decay 

SR-2547 

1 

75mm 

ISOmed 
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SR-2402 

1 

big  deep 

frag  shallow 

soil  +  frag 

SR-2601 

1 

37  or  81 

frag 

SR-2653 

1 

75  or  81 

ISOmed 

Pb  with  early  time  saturation  and 
depth 

SR-2515 

1 

155mm 

155mm 

SR-2519 

2 

ISO 

frag 

no  ISO?  Get  more  to  confirm 

SR-2091 

2 

IS0/IS02 

frag  x  5 

SR-2133 

2 

IS02 

frag 

gdt  half  way  b/w  2133-2143 

SR-2165 

2 

IS02 

frag,  =2166 

SR-2084 

2 

ISO2/60mm 

IS02 

SR-2088 

2 

ISO2/60mm 

IS02 

SR-2361 

2 

37mm 

37mm 

SR-2682 

2 

37mm 

37mm 

deep 

SR-2137 

2 

60/75 

frag 

SR-2077 

2 

60mm  body 

frag 

SR-2289 

2 

60mm  full? 

frag 

slower  decay  and  wider  L12  separation 

SR-2625 

2 

60mm/ISO2 

IS02+frag 

SR-2677 

2 

60mm  full? 

frag,  =2684 

faster  decay  and  wider  L12  sep 

SR-2413 

2 

105mm? 

105mm 

SR-2613 

3 

ISO 

frag 

offset 

SR-2206 

3 

ISO 

frag  x  3 

double  with  2213 

SR-2709 

3 

ISO 

frag  (=2704) 

most  lookalike+poor  coverage 

SR-2196 

3 

IS02 

IS02 

saturation? 

SR-2541 

3 

IS02 

frag  x  5 

offset 

SR-2610 

3 

37  or  IS02? 

frag 

offset 

SR-2022 

3 

37+stuff? 

37mm+frag 

SR-2410 

3 

37mm 

frag  (=2408) 

double  with  2408,  wide  L12  sep  though 

SR-2526 

3 

37mm 

37mm+frag 

saturation 

SR-2139 

3 

37mm 

37mm 

smaller  LI  or  faster  decay 

SR-2678 

3 

37mm 

37mm+frag 

faster  decay 

SR-2182 

3 

60mm  full? 

60mm 

L1L3  perfect,  L2  in  b/w 

SR-2500 

3 

75mm 

75mm 

saturation 

SR-2113 

3 

75mm 

75mm 

offset  and  deep 

4.  Parameters  and  thresholds 

We  apply  a  multi-stage  classifier  using  the  same  standard  methods  as  those  applied  to  other 
sensor  data  as  part  of  ESTCP  MR-201159.  The  classifier  is  primarily  based  on  polarizability 
library  misfits.  The  first  stage  uses  all  three  polarizabilities  (L1L2L3),  the  second  L1L2,  the  third 
L1L3  and  the  fourth  one  just  LI.  Transitions  from  one  stage  to  the  next  are  selected  by  applying 
the  classifier  to  the  entire  dataset  and  using  the  fact  that  some  items  were  marked  as  UXO  either 
during  the  inversion  QC  or  from  training  data.  For  each  stage  the  decision  statistic  resembles  an 
L-shaped  curve  in  which  the  elbow  marks  a  transition  in  the  statistic  -  stage  thresholds  are 
chosen  near  the  elbow  (Figure  36). 
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Figure  36:  Decision  statistic  (A)  based  on  "recognized  UXO"  (red  dots)  and  expected  ROC  curve  (B). 


5.  Stop-dig  decision 

A  dig  list  is  generated  with  the  multi-stage  classifier.  The  corresponding  polarizabilities  are 
visually  inspected  to  verify  that  the  classifier  performs  as  expected  and  to  gauge  the  relevance  of 
the  selected  parameters  -  for  instance  obvious  soil-like  models  may  be  selected  by  the  classifier 
because  they  resemble  a  37mm  or  105  mm  more  than  their  associated  clutter,  in  which  case  the 
model  can  be  manually  rejected.  Training  data  are  left  among  other  anomalies  to  relate  their 
associated  polarizabilities  to  similar-ranked  anomalies.  The  operator  manually  select  a  stop  dig 
point  after  which  UXO  are  less  likely  to  appear. 
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